[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:34:15 dist-test-slave-h3ft krb5kdc[9923](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:34:15 dist-test-slave-h3ft krb5kdc[9923](info): set up 2 sockets
Aug 07 08:34:15 dist-test-slave-h3ft krb5kdc[9923](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 08:34:17.118358 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.004s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:34:17 dist-test-slave-h3ft krb5kdc[9923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:34:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:34:17.283687 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:37145
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:37145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:17.441130 9941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:17.441722 9941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:17.441843 9941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:17.448369 9941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:17.448479 9941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:17.448514 9941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:17.448563 9941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:17.448613 9941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:17.456771 9941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:37145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:37145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:17.458796 9941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:17.460636 9941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:17.474974 9948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:17.475081 9941 server_base.cc:1034] running on GCE node
W20260807 08:34:17.475116 9951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:17.474980 9949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:17.480129 9941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:17.481962 9941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:17.483201 9941 hybrid_clock.cc:648] HybridClock initialized: now 1786091657483166 us; error 56 us; skew 500 ppm
Aug 07 08:34:17 dist-test-slave-h3ft krb5kdc[9923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091657, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:17.486848 9941 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:34:17.488057 9941 webserver.cc:460] Webserver started at http://127.9.174.62:40611/ using document root <none> and password file <none>
I20260807 08:34:17.488762 9941 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:17.488828 9941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:17.489061 9941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:17.491772 9941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "6a75ea874b6b4d1b805b97fb5075a2d3"
format_stamp: "Formatted at 2026-08-07 08:34:17 on dist-test-slave-h3ft"
server_key: "80f120423811834553c18223ae770bea"
server_key_iv: "a898f2b771ba1be1116d34d47c548ea0"
server_key_version: "encryptionkey@0"
I20260807 08:34:17.492528 9941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "6a75ea874b6b4d1b805b97fb5075a2d3"
format_stamp: "Formatted at 2026-08-07 08:34:17 on dist-test-slave-h3ft"
server_key: "80f120423811834553c18223ae770bea"
server_key_iv: "a898f2b771ba1be1116d34d47c548ea0"
server_key_version: "encryptionkey@0"
I20260807 08:34:17.497622 9941 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:34:17.500686 9962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:17.502184 9941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 08:34:17.502331 9941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "6a75ea874b6b4d1b805b97fb5075a2d3"
format_stamp: "Formatted at 2026-08-07 08:34:17 on dist-test-slave-h3ft"
server_key: "80f120423811834553c18223ae770bea"
server_key_iv: "a898f2b771ba1be1116d34d47c548ea0"
server_key_version: "encryptionkey@0"
I20260807 08:34:17.502463 9941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:17.515683 9941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:17.519923 9941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:17.520202 9941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:17.534094 9941 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:37145
I20260807 08:34:17.534129 10031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:37145 every 8 connection(s)
I20260807 08:34:17.535923 9941 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:17.537537 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 9941
I20260807 08:34:17.537770 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:34:17.539768 9912 external_mini_cluster.cc:1428] Setting key aadb0a68123ba96f79eba809845d21c0
I20260807 08:34:17.540352 10032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:17.548905 10032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3: Bootstrap starting.
I20260807 08:34:17.552181 10032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:17.553293 10032 log.cc:824] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:17.556797 10032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3: No bootstrap required, opened a new log
I20260807 08:34:17.561628 10032 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a75ea874b6b4d1b805b97fb5075a2d3" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37145 } }
I20260807 08:34:17.562001 10032 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:17.562098 10032 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a75ea874b6b4d1b805b97fb5075a2d3, State: Initialized, Role: FOLLOWER
I20260807 08:34:17.562808 10032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a75ea874b6b4d1b805b97fb5075a2d3" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37145 } }
I20260807 08:34:17.563050 10032 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:34:17.563133 10032 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:34:17.563256 10032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:17.565330 10032 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a75ea874b6b4d1b805b97fb5075a2d3" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37145 } }
I20260807 08:34:17.565897 10032 leader_election.cc:302] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a75ea874b6b4d1b805b97fb5075a2d3; no voters:
I20260807 08:34:17.566427 10032 leader_election.cc:288] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:34:17.566747 10040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:17.567255 10040 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [term 1 LEADER]: Becoming Leader. State: Replica: 6a75ea874b6b4d1b805b97fb5075a2d3, State: Running, Role: LEADER
I20260807 08:34:17.567786 10040 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a75ea874b6b4d1b805b97fb5075a2d3" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37145 } }
Aug 07 08:34:17 dist-test-slave-h3ft krb5kdc[9923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:17.568378 10032 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:17.570837 10043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a75ea874b6b4d1b805b97fb5075a2d3. Latest consensus state: current_term: 1 leader_uuid: "6a75ea874b6b4d1b805b97fb5075a2d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a75ea874b6b4d1b805b97fb5075a2d3" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37145 } } }
I20260807 08:34:17.571058 10043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:17.571575 10042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a75ea874b6b4d1b805b97fb5075a2d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a75ea874b6b4d1b805b97fb5075a2d3" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37145 } } }
I20260807 08:34:17.571805 10042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:17.578105 10035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:17.544110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50100 (local address 127.9.174.62:37145)
0807 08:34:17.544841 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:17.544890 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:17.546234 (+ 1344us) server_negotiation.cc:407] Connection header received
0807 08:34:17.552023 (+ 5789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:17.552050 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:17.552623 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:17.553111 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:17.557533 (+ 4422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:17.559419 (+ 1886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:17.561244 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:17.561962 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:17.562332 (+ 370us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:17.569343 (+ 7011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:17.569352 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:17.569386 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:17.569451 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:17.572982 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:17.573979 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:17.573988 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:17.573994 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:17.574196 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:17.574774 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:17.574780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:17.574782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:17.575374 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:17.575646 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:17.576412 (+ 766us) server_negotiation.cc:299] Negotiation successful
0807 08:34:17.576793 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":252,"threads_started":1}
W20260807 08:34:17.578526 10061 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:34:17.578658 10061 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:34:17.578819 10052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:17.580200 10052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:17.590803 10052 catalog_manager.cc:1347] Generated new cluster ID: c65b15401d3242efabab26497bb2715a
I20260807 08:34:17.590951 10052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:17.601042 10052 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:17.602377 10052 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:17.612282 10052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a75ea874b6b4d1b805b97fb5075a2d3: Generated new TSK 0
I20260807 08:34:17.613418 10052 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:34:17.663659 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37145
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:17.889016 10068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:17.889400 10068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:17.889454 10068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:17.894258 10068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:17.894361 10068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:17.894464 10068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:34:17.900952 10068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37145
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:17.902534 10068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:17.903980 10068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:17.919742 10075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:17.919811 10076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:17.919795 10081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:17.920208 10068 server_base.cc:1034] running on GCE node
I20260807 08:34:17.920893 10068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:17.921772 10068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:17.922992 10068 hybrid_clock.cc:648] HybridClock initialized: now 1786091657922972 us; error 34 us; skew 500 ppm
Aug 07 08:34:17 dist-test-slave-h3ft krb5kdc[9923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091657, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:17.927933 10068 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:34:17.929208 10068 webserver.cc:460] Webserver started at http://127.9.174.1:43849/ using document root <none> and password file <none>
I20260807 08:34:17.929989 10068 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:17.930099 10068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:17.930418 10068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:17.933604 10068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "e8ced64af63349a5aea53930a68be971"
format_stamp: "Formatted at 2026-08-07 08:34:17 on dist-test-slave-h3ft"
server_key: "1edb7bd3f89beed646ae84afd3ee5dae"
server_key_iv: "e2a2ec7ad0ef90811f331eb995f3638a"
server_key_version: "encryptionkey@0"
I20260807 08:34:17.934458 10068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "e8ced64af63349a5aea53930a68be971"
format_stamp: "Formatted at 2026-08-07 08:34:17 on dist-test-slave-h3ft"
server_key: "1edb7bd3f89beed646ae84afd3ee5dae"
server_key_iv: "e2a2ec7ad0ef90811f331eb995f3638a"
server_key_version: "encryptionkey@0"
I20260807 08:34:17.940202 10068 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 08:34:17.943696 10090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:17.945323 10068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 08:34:17.945525 10068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "e8ced64af63349a5aea53930a68be971"
format_stamp: "Formatted at 2026-08-07 08:34:17 on dist-test-slave-h3ft"
server_key: "1edb7bd3f89beed646ae84afd3ee5dae"
server_key_iv: "e2a2ec7ad0ef90811f331eb995f3638a"
server_key_version: "encryptionkey@0"
I20260807 08:34:17.945688 10068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:17.971208 10068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:17.976052 10068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:17.976327 10068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:17.977288 10068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:17.978536 10068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:17.978622 10068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:17.978710 10068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:17.978766 10068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:18.003399 10068 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:34217
I20260807 08:34:18.003520 10242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:34217 every 8 connection(s)
I20260807 08:34:18.004778 10068 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:34:18.008833 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 10068
I20260807 08:34:18.008982 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:34:18.009297 9912 external_mini_cluster.cc:1428] Setting key 34f151f9d2b1c4fc6c84ae85f9c47784
Aug 07 08:34:18 dist-test-slave-h3ft krb5kdc[9923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091657, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:18.026063 10035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:18.010421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:49699 (local address 127.9.174.62:37145)
0807 08:34:18.010590 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:18.010596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:18.010616 (+ 20us) server_negotiation.cc:407] Connection header received
0807 08:34:18.011857 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:18.011860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:18.011922 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:18.012006 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:18.013768 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:18.014760 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:18.016615 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:18.017057 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:18.017205 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:18.020887 (+ 3682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:18.020891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:18.020893 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:18.020923 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:18.022800 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:18.023780 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:18.023786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:18.023788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:18.023901 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:18.024527 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:18.024531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:18.024534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:18.024801 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:18.024931 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:18.025712 (+ 781us) server_negotiation.cc:299] Negotiation successful
0807 08:34:18.025886 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
I20260807 08:34:18.028123 10245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:18.008441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37145 (local address 127.9.174.1:49699)
0807 08:34:18.009923 (+ 1482us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:18.010013 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:18.011610 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:18.012234 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:18.012243 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:18.012861 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:18.013602 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:18.013620 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:18.014954 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:18.014958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:18.016469 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:18.016481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:18.017246 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:18.017249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:18.017476 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:18.018331 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:18.018352 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:18.020663 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:18.023216 (+ 2553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:18.023225 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:18.023260 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:18.023640 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:18.024015 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:18.024019 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:18.024021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:18.024378 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:18.024949 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:18.024960 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:18.025342 (+ 382us) client_negotiation.cc:769] Sending connection context
0807 08:34:18.026333 (+ 991us) client_negotiation.cc:240] Negotiation successful
0807 08:34:18.026795 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1121,"thread_start_us":806,"threads_started":1}
I20260807 08:34:18.030144 10243 heartbeater.cc:347] Connected to a master server at 127.9.174.62:37145
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
I20260807 08:34:18.030537 10243 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:18.031476 10243 heartbeater.cc:511] Master 127.9.174.62:37145 requested a full tablet report, sending...
I20260807 08:34:18.034070 9982 ts_manager.cc:194] Registered new tserver with Master: e8ced64af63349a5aea53930a68be971 (127.9.174.1:34217)
I20260807 08:34:18.035662 9982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:49699
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:34:18.052062 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37145
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:18.252653 10250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:18.253108 10250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:18.253202 10250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:18.258535 10250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:18.258702 10250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:18.258822 10250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:34:18.265729 10250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37145
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:18.267431 10250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:18.268936 10250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:18.283468 10257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:18.283408 10258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:18.283408 10262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:18.283880 10250 server_base.cc:1034] running on GCE node
I20260807 08:34:18.284505 10250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:18.285347 10250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:18.286521 10250 hybrid_clock.cc:648] HybridClock initialized: now 1786091658286504 us; error 45 us; skew 500 ppm
Aug 07 08:34:18 dist-test-slave-h3ft krb5kdc[9923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091658, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:18.290970 10250 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:34:18.292239 10250 webserver.cc:460] Webserver started at http://127.9.174.2:38115/ using document root <none> and password file <none>
I20260807 08:34:18.293036 10250 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:18.293107 10250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:18.293386 10250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:18.296303 10250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "44f3fbccb04948368405cc04d5cf0601"
format_stamp: "Formatted at 2026-08-07 08:34:18 on dist-test-slave-h3ft"
server_key: "749526ffd7094efddc66e710cce49e09"
server_key_iv: "def62efd7c0f135a46dd6daf3fd8f10f"
server_key_version: "encryptionkey@0"
I20260807 08:34:18.297122 10250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "44f3fbccb04948368405cc04d5cf0601"
format_stamp: "Formatted at 2026-08-07 08:34:18 on dist-test-slave-h3ft"
server_key: "749526ffd7094efddc66e710cce49e09"
server_key_iv: "def62efd7c0f135a46dd6daf3fd8f10f"
server_key_version: "encryptionkey@0"
I20260807 08:34:18.302268 10250 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 08:34:18.305552 10275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:18.307318 10250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 08:34:18.307459 10250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "44f3fbccb04948368405cc04d5cf0601"
format_stamp: "Formatted at 2026-08-07 08:34:18 on dist-test-slave-h3ft"
server_key: "749526ffd7094efddc66e710cce49e09"
server_key_iv: "def62efd7c0f135a46dd6daf3fd8f10f"
server_key_version: "encryptionkey@0"
I20260807 08:34:18.307598 10250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:18.326399 10250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:18.331291 10250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:18.331564 10250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:18.332485 10250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:18.333884 10250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:18.333966 10250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:18.334048 10250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:18.334100 10250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:18.359711 10250 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:42651
I20260807 08:34:18.359915 10433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:42651 every 8 connection(s)
I20260807 08:34:18.361248 10250 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:34:18.364811 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 10250
I20260807 08:34:18.364946 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:34:18.365346 9912 external_mini_cluster.cc:1428] Setting key 5ebf0cd5fd2364d7f64ccd3ae6ceb423
Aug 07 08:34:18 dist-test-slave-h3ft krb5kdc[9923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091658, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
I20260807 08:34:18.382537 10035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:18.364527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:50909 (local address 127.9.174.62:37145)
0807 08:34:18.364722 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:18.364726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:18.366593 (+ 1867us) server_negotiation.cc:407] Connection header received
0807 08:34:18.368029 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:18.368036 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:18.368110 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:18.368207 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:18.370086 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:18.371157 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:18.373438 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:18.373699 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:18.373805 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:18.378167 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:18.378171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:18.378173 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:18.378205 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:18.379965 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:18.380598 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:18.380602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:18.380604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:18.380702 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:18.381103 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:18.381108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:18.381110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:18.381391 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:18.381508 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:18.382154 (+ 646us) server_negotiation.cc:299] Negotiation successful
0807 08:34:18.382326 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 08:34:18.383980 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:18.365078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37145 (local address 127.9.174.2:50909)
0807 08:34:18.366398 (+ 1320us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:18.366477 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:18.367732 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:18.368410 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:18.368419 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:18.369063 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:18.369915 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:18.369936 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:18.371330 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:18.371335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:18.373273 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:18.373289 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:18.373902 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:18.373908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:18.374260 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:18.375146 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:18.375179 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:18.377936 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:18.380109 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:18.380116 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:18.380138 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:18.380479 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:18.380805 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:18.380809 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:18.380811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:18.380995 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:18.381506 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:18.381516 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:18.381867 (+ 351us) client_negotiation.cc:769] Sending connection context
0807 08:34:18.382110 (+ 243us) client_negotiation.cc:240] Negotiation successful
0807 08:34:18.382619 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1015,"thread_start_us":332,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
I20260807 08:34:18.385550 10434 heartbeater.cc:347] Connected to a master server at 127.9.174.62:37145
I20260807 08:34:18.385907 10434 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:18.386898 10434 heartbeater.cc:511] Master 127.9.174.62:37145 requested a full tablet report, sending...
I20260807 08:34:18.388638 9982 ts_manager.cc:194] Registered new tserver with Master: 44f3fbccb04948368405cc04d5cf0601 (127.9.174.2:42651)
I20260807 08:34:18.389351 9982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:50909
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:34:18.408692 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37145
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:18.603986 10440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:18.604347 10440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:18.604401 10440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:18.609360 10440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:18.609462 10440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:18.609570 10440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:34:18.616820 10440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37145
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:18.618408 10440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:18.619875 10440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:18.633890 10454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:18.633890 10451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:18.633863 10450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:18.634274 10440 server_base.cc:1034] running on GCE node
I20260807 08:34:18.634784 10440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:18.635708 10440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:18.636893 10440 hybrid_clock.cc:648] HybridClock initialized: now 1786091658636847 us; error 59 us; skew 500 ppm
Aug 07 08:34:18 dist-test-slave-h3ft krb5kdc[9923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091658, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:18.641255 10440 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:34:18.642470 10440 webserver.cc:460] Webserver started at http://127.9.174.3:46427/ using document root <none> and password file <none>
I20260807 08:34:18.643229 10440 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:18.643306 10440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:18.643527 10440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:18.646333 10440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "4c1d5435a9db4a11be096ff50119032d"
format_stamp: "Formatted at 2026-08-07 08:34:18 on dist-test-slave-h3ft"
server_key: "a38a38a0334aac15e3fa274addde3bba"
server_key_iv: "190bb8369a5443caf80a04cc516ec0e0"
server_key_version: "encryptionkey@0"
I20260807 08:34:18.647090 10440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "4c1d5435a9db4a11be096ff50119032d"
format_stamp: "Formatted at 2026-08-07 08:34:18 on dist-test-slave-h3ft"
server_key: "a38a38a0334aac15e3fa274addde3bba"
server_key_iv: "190bb8369a5443caf80a04cc516ec0e0"
server_key_version: "encryptionkey@0"
I20260807 08:34:18.652288 10440 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 08:34:18.655470 10465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:18.657042 10440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:34:18.657202 10440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "4c1d5435a9db4a11be096ff50119032d"
format_stamp: "Formatted at 2026-08-07 08:34:18 on dist-test-slave-h3ft"
server_key: "a38a38a0334aac15e3fa274addde3bba"
server_key_iv: "190bb8369a5443caf80a04cc516ec0e0"
server_key_version: "encryptionkey@0"
I20260807 08:34:18.657351 10440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:18.673547 10440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:18.678381 10440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:18.678714 10440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:18.679857 10440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:18.681129 10440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:18.681208 10440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:18.681290 10440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:18.681339 10440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:18.707159 10440 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:37301
I20260807 08:34:18.707392 10618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:37301 every 8 connection(s)
I20260807 08:34:18.708545 10440 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:34:18.712950 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 10440
I20260807 08:34:18.713165 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:34:18.713541 9912 external_mini_cluster.cc:1428] Setting key 89a0128a1960863fc9d00d60f7f41190
Aug 07 08:34:18 dist-test-slave-h3ft krb5kdc[9923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091658, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:18.730080 10035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:18.711724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:33143 (local address 127.9.174.62:37145)
0807 08:34:18.711936 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:18.711944 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:18.714644 (+ 2700us) server_negotiation.cc:407] Connection header received
0807 08:34:18.716331 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:18.716336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:18.716418 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:18.716521 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:18.718682 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:18.719462 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:18.720997 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:18.721259 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:18.721351 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:18.724964 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:18.724968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:18.724971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:18.725003 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:18.726845 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:18.727838 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:18.727843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:18.727844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:18.727964 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:18.728500 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:18.728503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:18.728505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:18.728825 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:18.728962 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:18.729683 (+ 721us) server_negotiation.cc:299] Negotiation successful
0807 08:34:18.729864 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 08:34:18.731637 10622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:18.712376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37145 (local address 127.9.174.3:33143)
0807 08:34:18.714423 (+ 2047us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:18.714503 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:18.716070 (+ 1567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:18.716741 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:18.716751 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:18.717639 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:18.718489 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:18.718512 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:18.719618 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:18.719621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:18.720856 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:18.720868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:18.721426 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:18.721429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:18.721640 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:18.722453 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:18.722473 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:18.724752 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:18.727206 (+ 2454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:18.727213 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:18.727241 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:18.727710 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:18.728105 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:18.728109 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:18.728111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:18.728354 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:18.728987 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:18.728996 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:18.729357 (+ 361us) client_negotiation.cc:769] Sending connection context
0807 08:34:18.729612 (+ 255us) client_negotiation.cc:240] Negotiation successful
0807 08:34:18.730109 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1673,"thread_start_us":459,"threads_started":1}
I20260807 08:34:18.733379 10619 heartbeater.cc:347] Connected to a master server at 127.9.174.62:37145
I20260807 08:34:18.733736 10619 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:18.734469 10619 heartbeater.cc:511] Master 127.9.174.62:37145 requested a full tablet report, sending...
I20260807 08:34:18.735878 9982 ts_manager.cc:194] Registered new tserver with Master: 4c1d5435a9db4a11be096ff50119032d (127.9.174.3:37301)
I20260807 08:34:18.736610 9982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:33143
I20260807 08:34:18.742270 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:34:18 dist-test-slave-h3ft krb5kdc[9923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091658, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:34:18 dist-test-slave-h3ft krb5kdc[9923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091658, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.174.1@KRBTEST.COM
I20260807 08:34:18.783680 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:18.770437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54534 (local address 127.9.174.1:34217)
0807 08:34:18.771083 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:18.771095 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:18.771143 (+ 48us) server_negotiation.cc:407] Connection header received
0807 08:34:18.771382 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:18.771389 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:18.771655 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:18.771818 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:18.773032 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:18.773937 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:18.775218 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:18.775688 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:18.775787 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:18.779292 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:18.779304 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:18.779319 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:18.779353 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:18.781366 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:18.781903 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:18.781910 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:18.781911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:18.782009 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:18.782350 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:18.782352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:18.782353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:18.782652 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:18.782840 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:18.783244 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 08:34:18.783453 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":335,"threads_started":1}
W20260807 08:34:18.785056 10127 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54534
Aug 07 08:34:18 dist-test-slave-h3ft krb5kdc[9923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:34:18 dist-test-slave-h3ft krb5kdc[9923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.1@KRBTEST.COM
I20260807 08:34:18.824218 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:18.814350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54542 (local address 127.9.174.1:34217)
0807 08:34:18.814635 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:18.814641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:18.814826 (+ 185us) server_negotiation.cc:407] Connection header received
0807 08:34:18.815054 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:18.815065 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:18.815163 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:18.815260 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:18.816188 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:18.816710 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:18.817682 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:18.817911 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:18.818032 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:18.821067 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:18.821070 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:18.821072 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:18.821100 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:18.822422 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:18.822901 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:18.822905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:18.822906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:18.822983 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:18.823309 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:18.823313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:18.823315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:18.823499 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:18.823647 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:18.823901 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 08:34:18.824030 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260807 08:34:18.825603 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 10068
I20260807 08:34:18.844862 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 10250
I20260807 08:34:18.859278 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 10440
I20260807 08:34:18.874308 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 9941
2026-08-07T08:34:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3127 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:34:18 dist-test-slave-h3ft krb5kdc[10636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:34:18 dist-test-slave-h3ft krb5kdc[10636](info): set up 2 sockets
Aug 07 08:34:18 dist-test-slave-h3ft krb5kdc[10636](info): commencing operation
krb5kdc: starting...
W20260807 08:34:20.196048 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:34:20 dist-test-slave-h3ft krb5kdc[10636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:34:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:34:20.350227 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:35381
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:35381
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:20.490942 10654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:20.491353 10654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:20.491405 10654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:20.495922 10654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:20.496016 10654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:20.496037 10654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:20.496054 10654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:20.496071 10654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:20.502271 10654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:35381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:35381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:20.503791 10654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:20.505148 10654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:20.516652 10669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:20.516652 10665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:20.516652 10666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:20.516987 10654 server_base.cc:1034] running on GCE node
I20260807 08:34:20.517800 10654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:20.519328 10654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:20.520524 10654 hybrid_clock.cc:648] HybridClock initialized: now 1786091660520510 us; error 33 us; skew 500 ppm
Aug 07 08:34:20 dist-test-slave-h3ft krb5kdc[10636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091660, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:20.524106 10654 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:34:20.525322 10654 webserver.cc:460] Webserver started at http://127.9.174.62:37307/ using document root <none> and password file <none>
I20260807 08:34:20.526039 10654 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:20.526109 10654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:20.526371 10654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:20.529034 10654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "8535d316e129477c8f2c5a0e4956bcfa"
format_stamp: "Formatted at 2026-08-07 08:34:20 on dist-test-slave-h3ft"
server_key: "fc9a91d812f44a0d68e57c5c8b7f4933"
server_key_iv: "6dbd3843ac2ca06f480651cb28123517"
server_key_version: "encryptionkey@0"
I20260807 08:34:20.529722 10654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "8535d316e129477c8f2c5a0e4956bcfa"
format_stamp: "Formatted at 2026-08-07 08:34:20 on dist-test-slave-h3ft"
server_key: "fc9a91d812f44a0d68e57c5c8b7f4933"
server_key_iv: "6dbd3843ac2ca06f480651cb28123517"
server_key_version: "encryptionkey@0"
I20260807 08:34:20.534755 10654 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:34:20.537830 10679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:20.539502 10654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:34:20.539686 10654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "8535d316e129477c8f2c5a0e4956bcfa"
format_stamp: "Formatted at 2026-08-07 08:34:20 on dist-test-slave-h3ft"
server_key: "fc9a91d812f44a0d68e57c5c8b7f4933"
server_key_iv: "6dbd3843ac2ca06f480651cb28123517"
server_key_version: "encryptionkey@0"
I20260807 08:34:20.539840 10654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:20.562100 10654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:20.566337 10654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:20.566630 10654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:20.575884 10654 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:35381
I20260807 08:34:20.575925 10753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:35381 every 8 connection(s)
I20260807 08:34:20.577224 10654 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:20.580480 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 10654
I20260807 08:34:20.580610 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:34:20.580981 9912 external_mini_cluster.cc:1428] Setting key d6b0bbf238de602742cf5676a1556319
I20260807 08:34:20.581425 10754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:20.590955 10754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa: Bootstrap starting.
Aug 07 08:34:20 dist-test-slave-h3ft krb5kdc[10636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:20.594527 10754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:20.595655 10754 log.cc:824] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:20.598652 10754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa: No bootstrap required, opened a new log
I20260807 08:34:20.599056 10759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:20.582898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60026 (local address 127.9.174.62:35381)
0807 08:34:20.583499 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:20.583525 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:20.583580 (+ 55us) server_negotiation.cc:407] Connection header received
0807 08:34:20.584570 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:20.584601 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:20.585138 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:20.585614 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:20.586390 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:20.588045 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:20.589018 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:20.589803 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:20.589988 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:20.593065 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:20.593072 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:20.593092 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:20.593133 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:20.595332 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:20.595810 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:20.595817 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:20.595823 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:20.595990 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:20.596324 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:20.596326 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:20.596327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:20.596907 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:20.597119 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:20.597464 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 08:34:20.597831 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":173,"threads_started":1}
I20260807 08:34:20.603492 10754 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535d316e129477c8f2c5a0e4956bcfa" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35381 } }
I20260807 08:34:20.603844 10754 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:20.603915 10754 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8535d316e129477c8f2c5a0e4956bcfa, State: Initialized, Role: FOLLOWER
I20260807 08:34:20.604465 10754 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535d316e129477c8f2c5a0e4956bcfa" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35381 } }
I20260807 08:34:20.604611 10754 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:34:20.604703 10754 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:34:20.604816 10754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:20.606149 10754 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535d316e129477c8f2c5a0e4956bcfa" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35381 } }
I20260807 08:34:20.606592 10754 leader_election.cc:302] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8535d316e129477c8f2c5a0e4956bcfa; no voters:
I20260807 08:34:20.607031 10754 leader_election.cc:288] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:34:20.607219 10761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:20.607506 10761 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [term 1 LEADER]: Becoming Leader. State: Replica: 8535d316e129477c8f2c5a0e4956bcfa, State: Running, Role: LEADER
I20260807 08:34:20.607937 10761 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535d316e129477c8f2c5a0e4956bcfa" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35381 } }
I20260807 08:34:20.608848 10754 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:20.610158 10763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8535d316e129477c8f2c5a0e4956bcfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535d316e129477c8f2c5a0e4956bcfa" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35381 } } }
I20260807 08:34:20.610329 10763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:20.610803 10779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:20.610803 10762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8535d316e129477c8f2c5a0e4956bcfa. Latest consensus state: current_term: 1 leader_uuid: "8535d316e129477c8f2c5a0e4956bcfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535d316e129477c8f2c5a0e4956bcfa" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35381 } } }
I20260807 08:34:20.611080 10762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:20.614832 10779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:20.622143 10779 catalog_manager.cc:1347] Generated new cluster ID: 88a70336f5df4e1891e2d8e35023894d
I20260807 08:34:20.622287 10779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:20.650385 10779 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:20.651759 10779 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:20.657295 10779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8535d316e129477c8f2c5a0e4956bcfa: Generated new TSK 0
I20260807 08:34:20.658398 10779 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:34:20.699646 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:35381
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:20.842329 10796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:20.842710 10796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:20.842767 10796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:20.850238 10796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:20.850366 10796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:20.850468 10796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:34:20.859316 10796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:35381
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:20.860981 10796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:20.862416 10796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:20.875167 10803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:20.875167 10804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:20.875167 10807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:20.875741 10796 server_base.cc:1034] running on GCE node
I20260807 08:34:20.876272 10796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:20.877146 10796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:20.878357 10796 hybrid_clock.cc:648] HybridClock initialized: now 1786091660878323 us; error 53 us; skew 500 ppm
Aug 07 08:34:20 dist-test-slave-h3ft krb5kdc[10636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091660, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:20.886485 10796 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:34:20.887864 10796 webserver.cc:460] Webserver started at http://127.9.174.1:35175/ using document root <none> and password file <none>
I20260807 08:34:20.888631 10796 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:20.888741 10796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:20.889034 10796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:20.891880 10796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "f7338c9c080d48c7b3df47766f7b9b2a"
format_stamp: "Formatted at 2026-08-07 08:34:20 on dist-test-slave-h3ft"
server_key: "9133f720f6c1bb975e467d4b1506f3f4"
server_key_iv: "0d72e2d6aa1da9be2b7364713ab3dc36"
server_key_version: "encryptionkey@0"
I20260807 08:34:20.892608 10796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "f7338c9c080d48c7b3df47766f7b9b2a"
format_stamp: "Formatted at 2026-08-07 08:34:20 on dist-test-slave-h3ft"
server_key: "9133f720f6c1bb975e467d4b1506f3f4"
server_key_iv: "0d72e2d6aa1da9be2b7364713ab3dc36"
server_key_version: "encryptionkey@0"
I20260807 08:34:20.897411 10796 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 08:34:20.900501 10819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:20.902045 10796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 08:34:20.902230 10796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "f7338c9c080d48c7b3df47766f7b9b2a"
format_stamp: "Formatted at 2026-08-07 08:34:20 on dist-test-slave-h3ft"
server_key: "9133f720f6c1bb975e467d4b1506f3f4"
server_key_iv: "0d72e2d6aa1da9be2b7364713ab3dc36"
server_key_version: "encryptionkey@0"
I20260807 08:34:20.902382 10796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:20.918359 10796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:20.922063 10796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:20.922314 10796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:20.923195 10796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:20.924239 10796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:20.924315 10796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:20.924438 10796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:20.924489 10796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:20.935387 10796 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:43459
I20260807 08:34:20.935457 10983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:43459 every 8 connection(s)
I20260807 08:34:20.936537 10796 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:34:20.945602 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 10796
I20260807 08:34:20.945729 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:34:20.946070 9912 external_mini_cluster.cc:1428] Setting key bb19dd0adceb91bd746c57613f2cd9de
Aug 07 08:34:20 dist-test-slave-h3ft krb5kdc[10636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091660, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:20.953493 10759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:20.939066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:55221 (local address 127.9.174.62:35381)
0807 08:34:20.939281 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:20.939287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:20.940424 (+ 1137us) server_negotiation.cc:407] Connection header received
0807 08:34:20.941988 (+ 1564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:20.941991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:20.942060 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:20.942148 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:20.943628 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:20.944309 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:20.945814 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:20.946069 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:20.946201 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:20.949454 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:20.949458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:20.949461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:20.949498 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:20.951131 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:20.951710 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:20.951714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:20.951716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:20.951807 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:20.952196 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:20.952200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:20.952202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:20.952410 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:20.952543 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:20.953149 (+ 606us) server_negotiation.cc:299] Negotiation successful
0807 08:34:20.953302 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 08:34:20.954582 10986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:20.939645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:35381 (local address 127.9.174.1:55221)
0807 08:34:20.940219 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:20.940273 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:20.941684 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:20.942358 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:20.942365 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:20.942808 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:20.943459 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:20.943476 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:20.944434 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:20.944438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:20.945612 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:20.945625 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:20.946260 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:20.946265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:20.946473 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:20.947335 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:20.947357 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:20.949266 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:20.951263 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:20.951269 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:20.951289 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:20.951595 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:20.951907 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:20.951910 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:20.951912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:20.952086 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:20.952537 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:20.952544 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:20.952865 (+ 321us) client_negotiation.cc:769] Sending connection context
0807 08:34:20.953090 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 08:34:20.953502 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":151,"threads_started":1}
I20260807 08:34:20.956180 10984 heartbeater.cc:347] Connected to a master server at 127.9.174.62:35381
I20260807 08:34:20.956501 10984 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:20.957182 10984 heartbeater.cc:511] Master 127.9.174.62:35381 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
I20260807 08:34:20.959465 10703 ts_manager.cc:194] Registered new tserver with Master: f7338c9c080d48c7b3df47766f7b9b2a (127.9.174.1:43459)
I20260807 08:34:20.960871 10703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:55221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:34:20.982087 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:35381
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:21.121295 10989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:21.121666 10989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:21.121750 10989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:21.126164 10989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:21.126245 10989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:21.126312 10989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:34:21.132400 10989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:35381
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:21.133894 10989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:21.135334 10989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:21.147727 11000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:21.147733 10998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:21.147733 10996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:21.148169 10989 server_base.cc:1034] running on GCE node
I20260807 08:34:21.148705 10989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:21.149506 10989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:21.150722 10989 hybrid_clock.cc:648] HybridClock initialized: now 1786091661150703 us; error 28 us; skew 500 ppm
Aug 07 08:34:21 dist-test-slave-h3ft krb5kdc[10636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091661, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:21.154156 10989 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:34:21.155333 10989 webserver.cc:460] Webserver started at http://127.9.174.2:42175/ using document root <none> and password file <none>
I20260807 08:34:21.156040 10989 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:21.156139 10989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:21.156440 10989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:21.158984 10989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "90d8fa706db948c697801fad05acf587"
format_stamp: "Formatted at 2026-08-07 08:34:21 on dist-test-slave-h3ft"
server_key: "1948925180266c364d2118aef3119745"
server_key_iv: "5b3e994f30463d5b9b8951c3bfb8b6b9"
server_key_version: "encryptionkey@0"
I20260807 08:34:21.159771 10989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "90d8fa706db948c697801fad05acf587"
format_stamp: "Formatted at 2026-08-07 08:34:21 on dist-test-slave-h3ft"
server_key: "1948925180266c364d2118aef3119745"
server_key_iv: "5b3e994f30463d5b9b8951c3bfb8b6b9"
server_key_version: "encryptionkey@0"
I20260807 08:34:21.164091 10989 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 08:34:21.166846 11008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:21.168315 10989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:34:21.168421 10989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "90d8fa706db948c697801fad05acf587"
format_stamp: "Formatted at 2026-08-07 08:34:21 on dist-test-slave-h3ft"
server_key: "1948925180266c364d2118aef3119745"
server_key_iv: "5b3e994f30463d5b9b8951c3bfb8b6b9"
server_key_version: "encryptionkey@0"
I20260807 08:34:21.168525 10989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:21.200145 10989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:21.204252 10989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:21.204452 10989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:21.205185 10989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:21.206178 10989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:21.206230 10989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:21.206275 10989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:21.206297 10989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:21.218098 10989 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:41361
I20260807 08:34:21.218119 11175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:41361 every 8 connection(s)
I20260807 08:34:21.219308 10989 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:34:21.221822 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 10989
I20260807 08:34:21.221961 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:34:21.222321 9912 external_mini_cluster.cc:1428] Setting key 3362b87baa0c461c670b3284d93bbd6f
Aug 07 08:34:21 dist-test-slave-h3ft krb5kdc[10636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091661, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
I20260807 08:34:21.238008 10759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.221838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:47383 (local address 127.9.174.62:35381)
0807 08:34:21.222020 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:21.222027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:21.223380 (+ 1353us) server_negotiation.cc:407] Connection header received
0807 08:34:21.225064 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:21.225071 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:21.225148 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:21.225252 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:21.226872 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.227526 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.229583 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.229808 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.229889 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.233781 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:21.233785 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:21.233787 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:21.233816 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:21.235620 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:21.236334 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:21.236337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:21.236338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:21.236425 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:21.236791 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:21.236793 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:21.236795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:21.236998 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:21.237082 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:21.237718 (+ 636us) server_negotiation.cc:299] Negotiation successful
0807 08:34:21.237845 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
I20260807 08:34:21.239259 11179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.222482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:35381 (local address 127.9.174.2:47383)
0807 08:34:21.223184 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:21.223255 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:21.224812 (+ 1557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:21.225462 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:21.225470 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:21.225937 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:21.226682 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:21.226713 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.227674 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.227681 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:21.229450 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:21.229468 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.229986 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.229993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:21.230306 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.231172 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:21.231200 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:21.233538 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:21.235768 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:21.235776 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:21.235796 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:21.236228 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:21.236539 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:21.236542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:21.236544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:21.236703 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:21.237099 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:21.237105 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:21.237463 (+ 358us) client_negotiation.cc:769] Sending connection context
0807 08:34:21.237664 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 08:34:21.237995 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":237,"threads_started":1}
I20260807 08:34:21.240885 11176 heartbeater.cc:347] Connected to a master server at 127.9.174.62:35381
I20260807 08:34:21.241228 11176 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:21.241899 11176 heartbeater.cc:511] Master 127.9.174.62:35381 requested a full tablet report, sending...
I20260807 08:34:21.243433 10703 ts_manager.cc:194] Registered new tserver with Master: 90d8fa706db948c697801fad05acf587 (127.9.174.2:41361)
I20260807 08:34:21.244517 10703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:47383
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:34:21.261214 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:35381
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:21.402920 11186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:21.403334 11186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:21.403429 11186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:21.407974 11186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:21.408069 11186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:21.408175 11186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:34:21.414318 11186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:35381
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:21.415807 11186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:21.417140 11186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:21.429243 11196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:21.429256 11192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:21.429345 11194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:21.429556 11186 server_base.cc:1034] running on GCE node
I20260807 08:34:21.430058 11186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:21.430859 11186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:21.432070 11186 hybrid_clock.cc:648] HybridClock initialized: now 1786091661432052 us; error 29 us; skew 500 ppm
Aug 07 08:34:21 dist-test-slave-h3ft krb5kdc[10636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091661, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:21.435678 11186 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:34:21.436933 11186 webserver.cc:460] Webserver started at http://127.9.174.3:39019/ using document root <none> and password file <none>
I20260807 08:34:21.437673 11186 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:21.437780 11186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:21.438082 11186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:21.440699 11186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "5111c16e133b4ac2bf1c303c8ed7582d"
format_stamp: "Formatted at 2026-08-07 08:34:21 on dist-test-slave-h3ft"
server_key: "a67eb10f27d9f6114c866b9cdbc5d540"
server_key_iv: "4a2f3b9bc64d63b6d3a581ecb6b1965f"
server_key_version: "encryptionkey@0"
I20260807 08:34:21.441397 11186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "5111c16e133b4ac2bf1c303c8ed7582d"
format_stamp: "Formatted at 2026-08-07 08:34:21 on dist-test-slave-h3ft"
server_key: "a67eb10f27d9f6114c866b9cdbc5d540"
server_key_iv: "4a2f3b9bc64d63b6d3a581ecb6b1965f"
server_key_version: "encryptionkey@0"
I20260807 08:34:21.446143 11186 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 08:34:21.449304 11205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:21.451120 11186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 08:34:21.451260 11186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "5111c16e133b4ac2bf1c303c8ed7582d"
format_stamp: "Formatted at 2026-08-07 08:34:21 on dist-test-slave-h3ft"
server_key: "a67eb10f27d9f6114c866b9cdbc5d540"
server_key_iv: "4a2f3b9bc64d63b6d3a581ecb6b1965f"
server_key_version: "encryptionkey@0"
I20260807 08:34:21.451408 11186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:21.481786 11186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:21.485618 11186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:21.485870 11186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:21.486685 11186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:21.487839 11186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:21.487910 11186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:21.487958 11186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:21.487974 11186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:21.499195 11186 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:45537
I20260807 08:34:21.499282 11374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:45537 every 8 connection(s)
I20260807 08:34:21.500380 11186 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:34:21.501250 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 11186
I20260807 08:34:21.501427 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:34:21.501848 9912 external_mini_cluster.cc:1428] Setting key 8c549b250df3dc3b66ac41b6f1efff6a
Aug 07 08:34:21 dist-test-slave-h3ft krb5kdc[10636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091661, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:21.518620 10759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.502786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:34943 (local address 127.9.174.62:35381)
0807 08:34:21.502974 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:21.502981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:21.504330 (+ 1349us) server_negotiation.cc:407] Connection header received
0807 08:34:21.506020 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:21.506024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:21.506095 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:21.506223 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:21.507883 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.508522 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.510041 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.510294 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.510430 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.513867 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:21.513871 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:21.513873 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:21.513906 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:21.515897 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:21.516660 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:21.516663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:21.516665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:21.516795 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:21.517379 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:21.517383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:21.517385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:21.517626 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:21.517710 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:21.518334 (+ 624us) server_negotiation.cc:299] Negotiation successful
0807 08:34:21.518464 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 08:34:21.519991 11380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.503494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:35381 (local address 127.9.174.3:34943)
0807 08:34:21.504161 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:21.504215 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:21.505709 (+ 1494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:21.506453 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:21.506461 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:21.506933 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:21.507704 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:21.507724 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.508683 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.508686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:21.509895 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:21.509914 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.510498 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.510504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:21.510740 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.511568 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:21.511589 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:21.513625 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:21.516042 (+ 2417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:21.516050 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:21.516072 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:21.516539 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:21.516965 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:21.516969 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:21.516970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:21.517258 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:21.517732 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:21.517739 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:21.518077 (+ 338us) client_negotiation.cc:769] Sending connection context
0807 08:34:21.518271 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 08:34:21.518651 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":203,"threads_started":1}
I20260807 08:34:21.521371 11376 heartbeater.cc:347] Connected to a master server at 127.9.174.62:35381
I20260807 08:34:21.521700 11376 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:21.522403 11376 heartbeater.cc:511] Master 127.9.174.62:35381 requested a full tablet report, sending...
I20260807 08:34:21.523720 10703 ts_manager.cc:194] Registered new tserver with Master: 5111c16e133b4ac2bf1c303c8ed7582d (127.9.174.3:45537)
I20260807 08:34:21.524428 10703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:34943
I20260807 08:34:21.529938 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:34:21.543390 10759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.534784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60036 (local address 127.9.174.62:35381)
0807 08:34:21.535077 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:21.535098 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:21.535235 (+ 137us) server_negotiation.cc:407] Connection header received
0807 08:34:21.535458 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:21.535462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:21.535515 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:21.535612 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:21.536582 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.537178 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.538080 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.538310 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.538431 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.540041 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:21.540045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:21.540047 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:21.540072 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:21.541458 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:21.541915 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:21.541919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:21.541923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:21.542015 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:21.542315 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:21.542318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:21.542320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:21.542594 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:21.542758 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:21.543045 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 08:34:21.543183 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260807 08:34:21.550004 10703 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:34:21.553886 10703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:34:21.572135 11391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.565018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:43459 (local address 127.0.0.1:55170)
0807 08:34:21.565655 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:21.565719 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:21.565903 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:21.567550 (+ 1647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:21.567556 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:21.567580 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:21.567697 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:21.567704 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.568821 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.568825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:21.570231 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:21.570242 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.571476 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.571480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:21.571593 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.571633 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 08:34:21.571784 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 08:34:21.571943 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":102,"spinlock_wait_cycles":7552,"thread_start_us":116,"threads_started":1}
I20260807 08:34:21.572201 11392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.565216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55170 (local address 127.9.174.1:43459)
0807 08:34:21.567114 (+ 1898us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:21.567121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:21.567143 (+ 22us) server_negotiation.cc:407] Connection header received
0807 08:34:21.567222 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:21.567226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:21.567403 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:21.567544 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:21.567831 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.568687 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.570374 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.571332 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.571465 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.571664 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:21.571835 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 08:34:21.571950 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1742,"thread_start_us":151,"threads_started":1}
I20260807 08:34:21.572973 11399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.566337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60328 (local address 127.9.174.3:45537)
0807 08:34:21.566972 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:21.566987 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:21.567043 (+ 56us) server_negotiation.cc:407] Connection header received
0807 08:34:21.567135 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:21.567139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:21.567331 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:21.567503 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:21.568033 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.569192 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.570817 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.571756 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.571867 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.572017 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:21.572645 (+ 628us) server_negotiation.cc:299] Negotiation successful
0807 08:34:21.572778 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":217,"threads_started":1}
I20260807 08:34:21.573019 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.565059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:41361 (local address 127.0.0.1:36476)
0807 08:34:21.565679 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:21.565719 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:21.565903 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:21.567248 (+ 1345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:21.567255 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:21.567280 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:21.567509 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:21.567522 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.569188 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.569191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:21.570659 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:21.570672 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.572090 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.572094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:21.572248 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.572278 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 08:34:21.572778 (+ 500us) client_negotiation.cc:240] Negotiation successful
0807 08:34:21.572862 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":91,"spinlock_wait_cycles":33536,"thread_start_us":134,"threads_started":1}
I20260807 08:34:21.573551 11393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.566076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:45537 (local address 127.0.0.1:60328)
0807 08:34:21.566560 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:21.566576 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:21.566684 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:21.567698 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:21.567701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:21.567727 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:21.567855 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:21.567862 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.569355 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.569360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:21.570665 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:21.570672 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.572090 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.572094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:21.572436 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.572456 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 08:34:21.572520 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:34:21.573412 (+ 892us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":290,"thread_start_us":136,"threads_started":1}
I20260807 08:34:21.573623 11397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.566100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36476 (local address 127.9.174.2:41361)
0807 08:34:21.566545 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:21.566560 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:21.566594 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:34:21.566717 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:21.566728 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:21.567064 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:21.567262 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:21.567665 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.568994 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.570818 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.571756 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.571867 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.572017 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:21.573329 (+ 1312us) server_negotiation.cc:299] Negotiation successful
0807 08:34:21.573437 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":125,"threads_started":1}
I20260807 08:34:21.576565 11277 tablet_service.cc:1467] Processing CreateTablet for tablet bcf3b31566ec4352a0404812762b34fb (DEFAULT_TABLE table=test-table [id=f674b70e5d6a4bfc905c02a46d4226a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:21.576568 11077 tablet_service.cc:1467] Processing CreateTablet for tablet bcf3b31566ec4352a0404812762b34fb (DEFAULT_TABLE table=test-table [id=f674b70e5d6a4bfc905c02a46d4226a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:21.576591 10885 tablet_service.cc:1467] Processing CreateTablet for tablet bcf3b31566ec4352a0404812762b34fb (DEFAULT_TABLE table=test-table [id=f674b70e5d6a4bfc905c02a46d4226a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:21.578122 10885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcf3b31566ec4352a0404812762b34fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:21.578122 11077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcf3b31566ec4352a0404812762b34fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:21.578123 11277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcf3b31566ec4352a0404812762b34fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:21.591128 11407 tablet_bootstrap.cc:492] T bcf3b31566ec4352a0404812762b34fb P 90d8fa706db948c697801fad05acf587: Bootstrap starting.
I20260807 08:34:21.591619 11403 tablet_bootstrap.cc:492] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d: Bootstrap starting.
I20260807 08:34:21.593706 11408 tablet_bootstrap.cc:492] T bcf3b31566ec4352a0404812762b34fb P f7338c9c080d48c7b3df47766f7b9b2a: Bootstrap starting.
I20260807 08:34:21.595412 11403 tablet_bootstrap.cc:654] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:21.596194 11407 tablet_bootstrap.cc:654] T bcf3b31566ec4352a0404812762b34fb P 90d8fa706db948c697801fad05acf587: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:21.597685 11403 log.cc:824] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:21.598049 11407 log.cc:824] T bcf3b31566ec4352a0404812762b34fb P 90d8fa706db948c697801fad05acf587: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:21.598511 11408 tablet_bootstrap.cc:654] T bcf3b31566ec4352a0404812762b34fb P f7338c9c080d48c7b3df47766f7b9b2a: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:21.600198 11408 log.cc:824] T bcf3b31566ec4352a0404812762b34fb P f7338c9c080d48c7b3df47766f7b9b2a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:21.600713 11403 tablet_bootstrap.cc:492] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d: No bootstrap required, opened a new log
I20260807 08:34:21.600983 11407 tablet_bootstrap.cc:492] T bcf3b31566ec4352a0404812762b34fb P 90d8fa706db948c697801fad05acf587: No bootstrap required, opened a new log
I20260807 08:34:21.600973 11403 ts_tablet_manager.cc:1397] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 08:34:21.601176 11407 ts_tablet_manager.cc:1397] T bcf3b31566ec4352a0404812762b34fb P 90d8fa706db948c697801fad05acf587: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 08:34:21.602396 11408 tablet_bootstrap.cc:492] T bcf3b31566ec4352a0404812762b34fb P f7338c9c080d48c7b3df47766f7b9b2a: No bootstrap required, opened a new log
I20260807 08:34:21.602617 11408 ts_tablet_manager.cc:1397] T bcf3b31566ec4352a0404812762b34fb P f7338c9c080d48c7b3df47766f7b9b2a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260807 08:34:21.608359 11403 raft_consensus.cc:348] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7338c9c080d48c7b3df47766f7b9b2a" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43459 } } peers { permanent_uuid: "90d8fa706db948c697801fad05acf587" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41361 } } peers { permanent_uuid: "5111c16e133b4ac2bf1c303c8ed7582d" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45537 } }
I20260807 08:34:21.608359 11407 raft_consensus.cc:348] T bcf3b31566ec4352a0404812762b34fb P 90d8fa706db948c697801fad05acf587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7338c9c080d48c7b3df47766f7b9b2a" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43459 } } peers { permanent_uuid: "90d8fa706db948c697801fad05acf587" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41361 } } peers { permanent_uuid: "5111c16e133b4ac2bf1c303c8ed7582d" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45537 } }
I20260807 08:34:21.608887 11407 raft_consensus.cc:374] T bcf3b31566ec4352a0404812762b34fb P 90d8fa706db948c697801fad05acf587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:21.608887 11403 raft_consensus.cc:374] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:21.609006 11403 raft_consensus.cc:729] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5111c16e133b4ac2bf1c303c8ed7582d, State: Initialized, Role: FOLLOWER
I20260807 08:34:21.609001 11407 raft_consensus.cc:729] T bcf3b31566ec4352a0404812762b34fb P 90d8fa706db948c697801fad05acf587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90d8fa706db948c697801fad05acf587, State: Initialized, Role: FOLLOWER
I20260807 08:34:21.609679 11403 consensus_queue.cc:260] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7338c9c080d48c7b3df47766f7b9b2a" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43459 } } peers { permanent_uuid: "90d8fa706db948c697801fad05acf587" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41361 } } peers { permanent_uuid: "5111c16e133b4ac2bf1c303c8ed7582d" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45537 } }
I20260807 08:34:21.609679 11407 consensus_queue.cc:260] T bcf3b31566ec4352a0404812762b34fb P 90d8fa706db948c697801fad05acf587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7338c9c080d48c7b3df47766f7b9b2a" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43459 } } peers { permanent_uuid: "90d8fa706db948c697801fad05acf587" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41361 } } peers { permanent_uuid: "5111c16e133b4ac2bf1c303c8ed7582d" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45537 } }
I20260807 08:34:21.609565 11408 raft_consensus.cc:348] T bcf3b31566ec4352a0404812762b34fb P f7338c9c080d48c7b3df47766f7b9b2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7338c9c080d48c7b3df47766f7b9b2a" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43459 } } peers { permanent_uuid: "90d8fa706db948c697801fad05acf587" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41361 } } peers { permanent_uuid: "5111c16e133b4ac2bf1c303c8ed7582d" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45537 } }
I20260807 08:34:21.609997 11408 raft_consensus.cc:374] T bcf3b31566ec4352a0404812762b34fb P f7338c9c080d48c7b3df47766f7b9b2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:21.610054 11408 raft_consensus.cc:729] T bcf3b31566ec4352a0404812762b34fb P f7338c9c080d48c7b3df47766f7b9b2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7338c9c080d48c7b3df47766f7b9b2a, State: Initialized, Role: FOLLOWER
I20260807 08:34:21.610709 11408 consensus_queue.cc:260] T bcf3b31566ec4352a0404812762b34fb P f7338c9c080d48c7b3df47766f7b9b2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7338c9c080d48c7b3df47766f7b9b2a" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43459 } } peers { permanent_uuid: "90d8fa706db948c697801fad05acf587" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41361 } } peers { permanent_uuid: "5111c16e133b4ac2bf1c303c8ed7582d" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45537 } }
I20260807 08:34:21.611297 11407 ts_tablet_manager.cc:1428] T bcf3b31566ec4352a0404812762b34fb P 90d8fa706db948c697801fad05acf587: Time spent starting tablet: real 0.010s user 0.007s sys 0.005s
I20260807 08:34:21.611297 11403 ts_tablet_manager.cc:1428] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d: Time spent starting tablet: real 0.010s user 0.007s sys 0.005s
I20260807 08:34:21.611754 11176 heartbeater.cc:503] Master 127.9.174.62:35381 was elected leader, sending a full tablet report...
I20260807 08:34:21.612035 11376 heartbeater.cc:503] Master 127.9.174.62:35381 was elected leader, sending a full tablet report...
I20260807 08:34:21.613941 11408 ts_tablet_manager.cc:1428] T bcf3b31566ec4352a0404812762b34fb P f7338c9c080d48c7b3df47766f7b9b2a: Time spent starting tablet: real 0.011s user 0.004s sys 0.006s
I20260807 08:34:21.614410 10984 heartbeater.cc:503] Master 127.9.174.62:35381 was elected leader, sending a full tablet report...
W20260807 08:34:21.688625 10985 tablet.cc:2369] T bcf3b31566ec4352a0404812762b34fb P f7338c9c080d48c7b3df47766f7b9b2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:34:21.721189 11178 tablet.cc:2369] T bcf3b31566ec4352a0404812762b34fb P 90d8fa706db948c697801fad05acf587: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:34:21.751869 11377 tablet.cc:2369] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:34:21.789068 11420 raft_consensus.cc:482] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:34:21.789529 11420 raft_consensus.cc:504] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7338c9c080d48c7b3df47766f7b9b2a" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43459 } } peers { permanent_uuid: "90d8fa706db948c697801fad05acf587" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41361 } } peers { permanent_uuid: "5111c16e133b4ac2bf1c303c8ed7582d" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45537 } }
I20260807 08:34:21.791460 11420 leader_election.cc:288] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7338c9c080d48c7b3df47766f7b9b2a (127.9.174.1:43459), 90d8fa706db948c697801fad05acf587 (127.9.174.2:41361)
I20260807 08:34:21.796851 11397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.792126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:38839 (local address 127.9.174.2:41361)
0807 08:34:21.792327 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:21.792333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:21.792357 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:34:21.792452 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:21.792456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:21.792523 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:21.792645 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:21.793355 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.794088 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.795468 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.796108 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.796239 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.796317 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:21.796636 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 08:34:21.796708 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 08:34:21.796870 11380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.791963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:41361 (local address 127.9.174.3:38839)
0807 08:34:21.792114 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:21.792134 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:21.792290 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:21.792843 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:21.792846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:21.792872 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:21.793195 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:21.793209 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.794222 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.794226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:21.795316 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:21.795327 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.796266 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.796269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:21.796474 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.796505 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 08:34:21.796610 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 08:34:21.796711 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 08:34:21.797849 11392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.792498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:45443 (local address 127.9.174.1:43459)
0807 08:34:21.792711 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:21.792716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:21.792846 (+ 130us) server_negotiation.cc:407] Connection header received
0807 08:34:21.792988 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:21.792991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:21.793082 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:21.793215 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:21.793708 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.794370 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.796253 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.797104 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.797260 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.797341 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:21.797637 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 08:34:21.797723 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260807 08:34:21.797936 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.791963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:43459 (local address 127.9.174.3:45443)
0807 08:34:21.792708 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:21.792720 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:21.792861 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:21.793233 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:21.793235 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:21.793250 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:21.793570 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:21.793579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.794500 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.794502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:21.796109 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:21.796120 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.797354 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.797358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:21.797487 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.797508 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 08:34:21.797614 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 08:34:21.797690 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":153,"thread_start_us":212,"threads_started":1}
I20260807 08:34:21.798033 11103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcf3b31566ec4352a0404812762b34fb" candidate_uuid: "5111c16e133b4ac2bf1c303c8ed7582d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90d8fa706db948c697801fad05acf587" is_pre_election: true
I20260807 08:34:21.798496 11103 raft_consensus.cc:2454] T bcf3b31566ec4352a0404812762b34fb P 90d8fa706db948c697801fad05acf587 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5111c16e133b4ac2bf1c303c8ed7582d in term 0.
I20260807 08:34:21.798628 10916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcf3b31566ec4352a0404812762b34fb" candidate_uuid: "5111c16e133b4ac2bf1c303c8ed7582d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7338c9c080d48c7b3df47766f7b9b2a" is_pre_election: true
I20260807 08:34:21.799038 10916 raft_consensus.cc:2454] T bcf3b31566ec4352a0404812762b34fb P f7338c9c080d48c7b3df47766f7b9b2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5111c16e133b4ac2bf1c303c8ed7582d in term 0.
I20260807 08:34:21.799320 11210 leader_election.cc:302] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5111c16e133b4ac2bf1c303c8ed7582d, 90d8fa706db948c697801fad05acf587; no voters:
I20260807 08:34:21.799685 11420 raft_consensus.cc:2781] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:34:21.799775 11420 raft_consensus.cc:482] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:34:21.799806 11420 raft_consensus.cc:3037] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:21.801359 11420 raft_consensus.cc:504] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7338c9c080d48c7b3df47766f7b9b2a" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43459 } } peers { permanent_uuid: "90d8fa706db948c697801fad05acf587" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41361 } } peers { permanent_uuid: "5111c16e133b4ac2bf1c303c8ed7582d" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45537 } }
I20260807 08:34:21.801828 11420 leader_election.cc:288] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [CANDIDATE]: Term 1 election: Requested vote from peers f7338c9c080d48c7b3df47766f7b9b2a (127.9.174.1:43459), 90d8fa706db948c697801fad05acf587 (127.9.174.2:41361)
I20260807 08:34:21.802412 10916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcf3b31566ec4352a0404812762b34fb" candidate_uuid: "5111c16e133b4ac2bf1c303c8ed7582d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7338c9c080d48c7b3df47766f7b9b2a"
I20260807 08:34:21.802459 11103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcf3b31566ec4352a0404812762b34fb" candidate_uuid: "5111c16e133b4ac2bf1c303c8ed7582d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90d8fa706db948c697801fad05acf587"
I20260807 08:34:21.802590 10916 raft_consensus.cc:3037] T bcf3b31566ec4352a0404812762b34fb P f7338c9c080d48c7b3df47766f7b9b2a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:21.802627 11103 raft_consensus.cc:3037] T bcf3b31566ec4352a0404812762b34fb P 90d8fa706db948c697801fad05acf587 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:21.804369 11103 raft_consensus.cc:2454] T bcf3b31566ec4352a0404812762b34fb P 90d8fa706db948c697801fad05acf587 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5111c16e133b4ac2bf1c303c8ed7582d in term 1.
I20260807 08:34:21.804512 10916 raft_consensus.cc:2454] T bcf3b31566ec4352a0404812762b34fb P f7338c9c080d48c7b3df47766f7b9b2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5111c16e133b4ac2bf1c303c8ed7582d in term 1.
I20260807 08:34:21.804824 11210 leader_election.cc:302] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5111c16e133b4ac2bf1c303c8ed7582d, 90d8fa706db948c697801fad05acf587; no voters:
I20260807 08:34:21.805054 11420 raft_consensus.cc:2781] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:21.805353 11420 raft_consensus.cc:686] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [term 1 LEADER]: Becoming Leader. State: Replica: 5111c16e133b4ac2bf1c303c8ed7582d, State: Running, Role: LEADER
I20260807 08:34:21.805804 11420 consensus_queue.cc:237] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7338c9c080d48c7b3df47766f7b9b2a" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43459 } } peers { permanent_uuid: "90d8fa706db948c697801fad05acf587" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41361 } } peers { permanent_uuid: "5111c16e133b4ac2bf1c303c8ed7582d" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45537 } }
I20260807 08:34:21.810772 10703 catalog_manager.cc:5495] T bcf3b31566ec4352a0404812762b34fb P 5111c16e133b4ac2bf1c303c8ed7582d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5111c16e133b4ac2bf1c303c8ed7582d (127.9.174.3). New cstate: current_term: 1 leader_uuid: "5111c16e133b4ac2bf1c303c8ed7582d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7338c9c080d48c7b3df47766f7b9b2a" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90d8fa706db948c697801fad05acf587" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5111c16e133b4ac2bf1c303c8ed7582d" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45537 } health_report { overall_health: HEALTHY } } }
Aug 07 08:34:21 dist-test-slave-h3ft krb5kdc[10636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.1@KRBTEST.COM
I20260807 08:34:21.885157 11392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.871983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55182 (local address 127.9.174.1:43459)
0807 08:34:21.872209 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:21.872215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:21.872888 (+ 673us) server_negotiation.cc:407] Connection header received
0807 08:34:21.873094 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:21.873101 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:21.873189 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:21.873291 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:21.874436 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.875150 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.876349 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.876613 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.876729 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.879874 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:21.879932 (+ 58us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:21.879956 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:21.880006 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:21.882802 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:21.883484 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:21.883491 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:21.883493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:21.883609 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:21.883954 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:21.883957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:21.883958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:21.884311 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:21.884530 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:21.884822 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 08:34:21.884980 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 08:34:21 dist-test-slave-h3ft krb5kdc[10636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091661, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:34:21 dist-test-slave-h3ft krb5kdc[10636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091661, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.174.1@KRBTEST.COM
I20260807 08:34:21.925216 11392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.913912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55198 (local address 127.9.174.1:43459)
0807 08:34:21.914118 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:21.914124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:21.914292 (+ 168us) server_negotiation.cc:407] Connection header received
0807 08:34:21.914541 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:21.914544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:21.914599 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:21.914705 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:21.915705 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.916417 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.917418 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.917753 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.917889 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.921315 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:21.921319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:21.921321 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:21.921362 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:21.923169 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:21.923666 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:21.923671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:21.923673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:21.923774 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:21.924098 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:21.924102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:21.924104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:21.924325 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:21.924494 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:21.924834 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 08:34:21.924996 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
Aug 07 08:34:21 dist-test-slave-h3ft krb5kdc[10636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:34:21 dist-test-slave-h3ft krb5kdc[10636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.1@KRBTEST.COM
I20260807 08:34:21.964900 11392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:21.953776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55206 (local address 127.9.174.1:43459)
0807 08:34:21.954033 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:21.954040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:21.954255 (+ 215us) server_negotiation.cc:407] Connection header received
0807 08:34:21.954406 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:21.954410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:21.954477 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:21.954583 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:21.955504 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.956100 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.957274 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:21.957529 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:21.957648 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:21.960988 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:21.960993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:21.960996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:21.961034 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:21.962857 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:21.963424 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:21.963427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:21.963429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:21.963522 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:21.963825 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:21.963828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:21.963829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:21.964053 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:21.964211 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:21.964560 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 08:34:21.964751 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260807 08:34:21.966840 10860 tablet_service.cc:2908] Rejecting scan request for tablet bcf3b31566ec4352a0404812762b34fb: Uninitialized: clean time has not yet been initialized
I20260807 08:34:21.969651 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 10796
I20260807 08:34:21.986481 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 10989
I20260807 08:34:22.004078 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 11186
I20260807 08:34:22.020593 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 10654
2026-08-07T08:34:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (3143 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:34:22 dist-test-slave-h3ft krb5kdc[11440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:34:22 dist-test-slave-h3ft krb5kdc[11440](info): set up 2 sockets
Aug 07 08:34:22 dist-test-slave-h3ft krb5kdc[11440](info): commencing operation
krb5kdc: starting...
W20260807 08:34:23.307276 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:34:23 dist-test-slave-h3ft krb5kdc[11440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:34:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:34:23.476653 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:35827
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:46441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:35827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:23.625340 11455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:23.625795 11455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:23.625878 11455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:23.630321 11455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:23.630450 11455 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 08:34:23.630484 11455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:23.630523 11455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:23.630575 11455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:23.630613 11455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:23.638283 11455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:46441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:35827
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:35827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:23.640131 11455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:23.641695 11455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:23.653651 11461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:23.653664 11462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:23.653787 11465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:23.654352 11455 server_base.cc:1034] running on GCE node
I20260807 08:34:23.655287 11455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:23.657049 11455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:23.658319 11455 hybrid_clock.cc:648] HybridClock initialized: now 1786091663658300 us; error 46 us; skew 500 ppm
Aug 07 08:34:23 dist-test-slave-h3ft krb5kdc[11440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091663, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:23.662216 11455 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:34:23.663607 11455 webserver.cc:460] Webserver started at http://127.9.174.62:39461/ using document root <none> and password file <none>
I20260807 08:34:23.664348 11455 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:23.664422 11455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:23.664651 11455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:23.667515 11455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "1cfc7bd65b2a44b19144d543f6041d12"
format_stamp: "Formatted at 2026-08-07 08:34:23 on dist-test-slave-h3ft"
server_key: "52097106da1dc180503757c09e3cb5d9"
server_key_iv: "e2ad18bdbd501f151fa35fb10041e157"
server_key_version: "encryptionkey@0"
I20260807 08:34:23.668226 11455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "1cfc7bd65b2a44b19144d543f6041d12"
format_stamp: "Formatted at 2026-08-07 08:34:23 on dist-test-slave-h3ft"
server_key: "52097106da1dc180503757c09e3cb5d9"
server_key_iv: "e2ad18bdbd501f151fa35fb10041e157"
server_key_version: "encryptionkey@0"
I20260807 08:34:23.673672 11455 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:34:23.677270 11476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:23.679438 11455 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 08:34:23.679651 11455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "1cfc7bd65b2a44b19144d543f6041d12"
format_stamp: "Formatted at 2026-08-07 08:34:23 on dist-test-slave-h3ft"
server_key: "52097106da1dc180503757c09e3cb5d9"
server_key_iv: "e2ad18bdbd501f151fa35fb10041e157"
server_key_version: "encryptionkey@0"
I20260807 08:34:23.679816 11455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:23.698886 11455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:23.703652 11455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:23.703926 11455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:23.715355 11455 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:35827
I20260807 08:34:23.715377 11569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:35827 every 8 connection(s)
I20260807 08:34:23.716812 11455 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:23.718719 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 11455
I20260807 08:34:23.718842 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:34:23.719242 9912 external_mini_cluster.cc:1428] Setting key 78235b2cf037ebaa7a1d7deab4169ff3
I20260807 08:34:23.721390 11571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:23.729228 11571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12: Bootstrap starting.
Aug 07 08:34:23 dist-test-slave-h3ft krb5kdc[11440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:23.732858 11571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:23.734359 11571 log.cc:824] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:23.736827 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:23.721198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39110 (local address 127.9.174.62:35827)
0807 08:34:23.721795 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:23.721815 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:23.721863 (+ 48us) server_negotiation.cc:407] Connection header received
0807 08:34:23.722722 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:23.722745 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:23.723298 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:23.723771 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:23.724747 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:23.725906 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:23.726866 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:23.727481 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:23.727668 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:23.730784 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:23.730790 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:23.730808 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:23.730843 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:23.733108 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:23.733676 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:23.733688 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:23.733694 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:23.733846 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:23.734169 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:23.734173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:23.734175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:23.734766 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:23.734993 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:23.735386 (+ 393us) server_negotiation.cc:299] Negotiation successful
0807 08:34:23.735832 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":202,"threads_started":1}
I20260807 08:34:23.737293 11571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12: No bootstrap required, opened a new log
I20260807 08:34:23.741283 11571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cfc7bd65b2a44b19144d543f6041d12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35827 } }
I20260807 08:34:23.741676 11571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:23.741776 11571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cfc7bd65b2a44b19144d543f6041d12, State: Initialized, Role: FOLLOWER
I20260807 08:34:23.742477 11571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cfc7bd65b2a44b19144d543f6041d12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35827 } }
I20260807 08:34:23.742684 11571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:34:23.742794 11571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:34:23.742947 11571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:23.744374 11571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cfc7bd65b2a44b19144d543f6041d12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35827 } }
I20260807 08:34:23.744904 11571 leader_election.cc:302] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cfc7bd65b2a44b19144d543f6041d12; no voters:
I20260807 08:34:23.745396 11571 leader_election.cc:288] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:34:23.745642 11580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:23.745925 11580 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [term 1 LEADER]: Becoming Leader. State: Replica: 1cfc7bd65b2a44b19144d543f6041d12, State: Running, Role: LEADER
I20260807 08:34:23.746364 11580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cfc7bd65b2a44b19144d543f6041d12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35827 } }
I20260807 08:34:23.747452 11571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:23.748903 11584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cfc7bd65b2a44b19144d543f6041d12. Latest consensus state: current_term: 1 leader_uuid: "1cfc7bd65b2a44b19144d543f6041d12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cfc7bd65b2a44b19144d543f6041d12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35827 } } }
I20260807 08:34:23.748929 11583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cfc7bd65b2a44b19144d543f6041d12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cfc7bd65b2a44b19144d543f6041d12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35827 } } }
I20260807 08:34:23.749135 11583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:23.749135 11584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:23.749704 11595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:23.753822 11595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:23.761237 11595 catalog_manager.cc:1347] Generated new cluster ID: 1b303cc9c0824616beadcd4cb56847ef
I20260807 08:34:23.761363 11595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:23.798707 11595 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:23.800108 11595 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:23.810566 11595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12: Generated new TSK 0
I20260807 08:34:23.811789 11595 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:34:23.856153 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:35827
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:46441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:24.004916 11608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:24.005307 11608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:24.005363 11608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:24.009910 11608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:24.010015 11608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:24.010061 11608 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:34:24.010133 11608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:34:24.016697 11608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:46441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.174.62:35827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:24.018309 11608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:24.019764 11608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:24.032341 11617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:24.032341 11614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:24.032648 11608 server_base.cc:1034] running on GCE node
W20260807 08:34:24.032362 11615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:24.033383 11608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:24.034314 11608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:24.035607 11608 hybrid_clock.cc:648] HybridClock initialized: now 1786091664035544 us; error 74 us; skew 500 ppm
Aug 07 08:34:24 dist-test-slave-h3ft krb5kdc[11440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091664, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:24.039480 11608 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:34:24.040820 11608 webserver.cc:460] Webserver started at http://127.9.174.1:38565/ using document root <none> and password file <none>
I20260807 08:34:24.041584 11608 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:24.041656 11608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:24.041950 11608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:24.044714 11608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "50dcad7d4e944899830bf985c9f0d464"
format_stamp: "Formatted at 2026-08-07 08:34:24 on dist-test-slave-h3ft"
server_key: "6441780c264ed6803d5f9c12fef28e95"
server_key_iv: "37bc2773f52cda9722bc9998e045ffda"
server_key_version: "encryptionkey@0"
I20260807 08:34:24.045428 11608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "50dcad7d4e944899830bf985c9f0d464"
format_stamp: "Formatted at 2026-08-07 08:34:24 on dist-test-slave-h3ft"
server_key: "6441780c264ed6803d5f9c12fef28e95"
server_key_iv: "37bc2773f52cda9722bc9998e045ffda"
server_key_version: "encryptionkey@0"
I20260807 08:34:24.050961 11608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:34:24.054965 11624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:24.056734 11608 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:34:24.056936 11608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "50dcad7d4e944899830bf985c9f0d464"
format_stamp: "Formatted at 2026-08-07 08:34:24 on dist-test-slave-h3ft"
server_key: "6441780c264ed6803d5f9c12fef28e95"
server_key_iv: "37bc2773f52cda9722bc9998e045ffda"
server_key_version: "encryptionkey@0"
I20260807 08:34:24.057077 11608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:24.073904 11608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:24.078058 11608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:24.078275 11608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:24.080292 11608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:24.080354 11608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:24.080430 11608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:24.080466 11608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:34:24 dist-test-slave-h3ft krb5kdc[11440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091664, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:24.094051 11608 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:36385
I20260807 08:34:24.094193 11785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:36385 every 8 connection(s)
I20260807 08:34:24.095521 11608 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:34:24.096966 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 11608
I20260807 08:34:24.097121 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:34:24.097468 9912 external_mini_cluster.cc:1428] Setting key 4e6b52260c64fcaa1775b638d4d8a4bf
I20260807 08:34:24.098259 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.081570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:51307 (local address 127.9.174.62:35827)
0807 08:34:24.081733 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:24.081738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:24.083181 (+ 1443us) server_negotiation.cc:407] Connection header received
0807 08:34:24.084656 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:24.084663 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:24.084753 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:24.084876 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:24.086952 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.087990 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.089503 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.089775 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.089909 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.093472 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:24.093476 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:24.093479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:24.093519 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:24.095519 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:24.096186 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:24.096190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:24.096192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:24.096353 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:24.096787 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:24.096791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:24.096793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:24.097099 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:24.097212 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:24.097844 (+ 632us) server_negotiation.cc:299] Negotiation successful
0807 08:34:24.098052 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:34:24.099464 11638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.082282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:35827 (local address 127.9.174.1:51307)
0807 08:34:24.082902 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:24.082987 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:24.084310 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:24.085136 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:24.085147 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:24.085727 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:24.086705 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.086733 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.088173 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.088177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.089333 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.089345 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.089990 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.089995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.090211 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.090987 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:24.091055 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:24.093246 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:24.095645 (+ 2399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:24.095650 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:24.095664 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:24.096062 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:24.096469 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:24.096472 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:24.096474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:24.096664 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:24.097223 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:24.097231 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:24.097557 (+ 326us) client_negotiation.cc:769] Sending connection context
0807 08:34:24.097757 (+ 200us) client_negotiation.cc:240] Negotiation successful
0807 08:34:24.098179 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":169,"threads_started":1}
I20260807 08:34:24.101128 11786 heartbeater.cc:347] Connected to a master server at 127.9.174.62:35827
I20260807 08:34:24.101311 11786 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:24.102046 11786 heartbeater.cc:511] Master 127.9.174.62:35827 requested a full tablet report, sending...
I20260807 08:34:24.104753 11502 ts_manager.cc:194] Registered new tserver with Master: 50dcad7d4e944899830bf985c9f0d464 (127.9.174.1:36385)
I20260807 08:34:24.106695 11502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:51307
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
I20260807 08:34:24.113814 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.103984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:53343 (local address 127.9.174.62:35827)
0807 08:34:24.104140 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:24.104144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:24.104424 (+ 280us) server_negotiation.cc:407] Connection header received
0807 08:34:24.104579 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:24.104582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:24.104630 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:24.104722 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:24.105268 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.105857 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.106959 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.107273 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.107388 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.110242 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:24.110246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:24.110250 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:24.110284 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:24.111952 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:24.112417 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:24.112421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:24.112423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:24.112517 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:24.112854 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:24.112859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:24.112860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:24.113066 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:24.113182 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:24.113465 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 08:34:24.113614 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 08:34:24.113821 11799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.103928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:35827 (local address 127.9.174.1:53343)
0807 08:34:24.104331 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:24.104344 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:24.104469 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:24.104741 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:24.104744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:24.105008 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:24.105147 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.105154 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.105996 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.106000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.106739 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.106749 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.107399 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.107402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.107510 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.109644 (+ 2134us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:24.109658 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:24.110065 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:24.112077 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:24.112081 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:24.112083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:24.112310 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:24.112657 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:24.112660 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:24.112663 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:24.112758 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:24.113169 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:24.113172 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:24.113285 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 08:34:24.113396 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 08:34:24.113582 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":154,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:34:24.136018 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:35827
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:46441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:24.274588 11801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:24.275000 11801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:24.275118 11801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:24.279402 11801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:24.279531 11801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:24.279610 11801 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:34:24.279697 11801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:34:24.285962 11801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:46441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.174.62:35827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:24.287477 11801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:24.288834 11801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:34:24.301030 11801 server_base.cc:1034] running on GCE node
W20260807 08:34:24.300868 11807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:24.300870 11811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:24.300957 11808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:24.301783 11801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:24.302626 11801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:24.303830 11801 hybrid_clock.cc:648] HybridClock initialized: now 1786091664303774 us; error 68 us; skew 500 ppm
Aug 07 08:34:24 dist-test-slave-h3ft krb5kdc[11440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091664, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:24.307268 11801 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:34:24.308398 11801 webserver.cc:460] Webserver started at http://127.9.174.2:36967/ using document root <none> and password file <none>
I20260807 08:34:24.309105 11801 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:24.309206 11801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:24.309542 11801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:24.312242 11801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "a811d7866acb431983af7856dbc1b1a9"
format_stamp: "Formatted at 2026-08-07 08:34:24 on dist-test-slave-h3ft"
server_key: "3e60668c2cdaad6944b03d0bf3124e4a"
server_key_iv: "6d9ae1ca1c85e5ab8e81d3d6091eaf5f"
server_key_version: "encryptionkey@0"
I20260807 08:34:24.312956 11801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "a811d7866acb431983af7856dbc1b1a9"
format_stamp: "Formatted at 2026-08-07 08:34:24 on dist-test-slave-h3ft"
server_key: "3e60668c2cdaad6944b03d0bf3124e4a"
server_key_iv: "6d9ae1ca1c85e5ab8e81d3d6091eaf5f"
server_key_version: "encryptionkey@0"
I20260807 08:34:24.319141 11801 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 08:34:24.322837 11820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:24.324604 11801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:34:24.324810 11801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "a811d7866acb431983af7856dbc1b1a9"
format_stamp: "Formatted at 2026-08-07 08:34:24 on dist-test-slave-h3ft"
server_key: "3e60668c2cdaad6944b03d0bf3124e4a"
server_key_iv: "6d9ae1ca1c85e5ab8e81d3d6091eaf5f"
server_key_version: "encryptionkey@0"
I20260807 08:34:24.324970 11801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:24.344913 11801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:24.349454 11801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:24.349722 11801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:24.351904 11801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:24.351991 11801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:24.352077 11801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:24.352123 11801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:34:24 dist-test-slave-h3ft krb5kdc[11440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091664, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:24.366593 11801 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:33079
I20260807 08:34:24.366677 11977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:33079 every 8 connection(s)
I20260807 08:34:24.367898 11801 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:34:24.369916 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.353124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:35017 (local address 127.9.174.62:35827)
0807 08:34:24.353301 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:24.353305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:24.354615 (+ 1310us) server_negotiation.cc:407] Connection header received
0807 08:34:24.356106 (+ 1491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:24.356111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:24.356211 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:24.356336 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:24.358075 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.358688 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.360336 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.360703 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.360838 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.365093 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:24.365097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:24.365099 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:24.365138 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:24.367177 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:24.367789 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:24.367793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:24.367795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:24.367891 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:24.368301 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:24.368305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:24.368306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:24.368620 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:24.368722 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:24.369599 (+ 877us) server_negotiation.cc:299] Negotiation successful
0807 08:34:24.369757 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 08:34:24.370960 11831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.353716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:35827 (local address 127.9.174.2:35017)
0807 08:34:24.354410 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:24.354464 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:24.355706 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:24.356623 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:24.356633 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:24.357209 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:24.357891 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.357917 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.358835 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.358839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.360182 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.360194 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.360936 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.360941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.361203 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.362203 (+ 1000us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:24.362237 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:24.364774 (+ 2537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:24.367322 (+ 2548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:24.367329 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:24.367348 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:24.367660 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:24.367996 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:24.368000 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:24.368005 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:24.368189 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:24.368962 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:24.368969 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:24.369302 (+ 333us) client_negotiation.cc:769] Sending connection context
0807 08:34:24.369502 (+ 200us) client_negotiation.cc:240] Negotiation successful
0807 08:34:24.369844 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":191,"threads_started":1}
I20260807 08:34:24.372681 11978 heartbeater.cc:347] Connected to a master server at 127.9.174.62:35827
I20260807 08:34:24.372879 11978 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:24.373678 11978 heartbeater.cc:511] Master 127.9.174.62:35827 requested a full tablet report, sending...
I20260807 08:34:24.375229 11501 ts_manager.cc:194] Registered new tserver with Master: a811d7866acb431983af7856dbc1b1a9 (127.9.174.2:33079)
I20260807 08:34:24.376142 11501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:35017
I20260807 08:34:24.377331 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 11801
I20260807 08:34:24.377486 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:34:24.377808 9912 external_mini_cluster.cc:1428] Setting key 144a4ca606f087436e9a1721d9386460
I20260807 08:34:24.385133 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.375807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:41121 (local address 127.9.174.62:35827)
0807 08:34:24.375980 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:24.375986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:24.376263 (+ 277us) server_negotiation.cc:407] Connection header received
0807 08:34:24.376375 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:24.376377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:24.376440 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:24.376543 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:24.377197 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.377975 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.379063 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.379388 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.379534 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.381088 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:24.381091 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:24.381094 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:24.381119 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:24.382966 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:24.383582 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:24.383585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:24.383586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:24.383674 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:24.383992 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:24.383995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:24.383996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:24.384246 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:24.384331 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:24.384771 (+ 440us) server_negotiation.cc:299] Negotiation successful
0807 08:34:24.384970 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 08:34:24.385195 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.375731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:35827 (local address 127.9.174.2:41121)
0807 08:34:24.376129 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:24.376147 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:24.376276 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:24.376564 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:24.376568 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:24.376821 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:24.377046 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.377057 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.378150 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.378154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.378907 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.378924 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.379556 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.379565 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.379664 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.380309 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:24.380332 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:24.380926 (+ 594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:24.383135 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:24.383142 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:24.383146 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:24.383460 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:24.383769 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:24.383775 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:24.383777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:24.383894 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:24.384350 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:24.384355 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:24.384500 (+ 145us) client_negotiation.cc:769] Sending connection context
0807 08:34:24.384648 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 08:34:24.384858 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":172,"threads_started":1}
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:34:24.415802 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:35827
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:46441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:24.556391 11991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:24.556733 11991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:24.556785 11991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:24.561022 11991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:24.561085 11991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:24.561126 11991 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:34:24.561160 11991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:34:24.567040 11991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:46441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.174.62:35827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:24.568432 11991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:24.569729 11991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:24.581779 12002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:24.581772 11998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:24.581775 11997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:24.582248 11991 server_base.cc:1034] running on GCE node
I20260807 08:34:24.582748 11991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:24.583604 11991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:24.584766 11991 hybrid_clock.cc:648] HybridClock initialized: now 1786091664584729 us; error 47 us; skew 500 ppm
Aug 07 08:34:24 dist-test-slave-h3ft krb5kdc[11440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091664, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:24.588229 11991 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:34:24.589326 11991 webserver.cc:460] Webserver started at http://127.9.174.3:44369/ using document root <none> and password file <none>
I20260807 08:34:24.590042 11991 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:24.590111 11991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:24.590401 11991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:24.593055 11991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "829561e191174809a66563f24e24a3d2"
format_stamp: "Formatted at 2026-08-07 08:34:24 on dist-test-slave-h3ft"
server_key: "c8233856d7b15886d0b4144245f41dbf"
server_key_iv: "547a3df4ab1f7016d1b68e300278b4cc"
server_key_version: "encryptionkey@0"
I20260807 08:34:24.593720 11991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "829561e191174809a66563f24e24a3d2"
format_stamp: "Formatted at 2026-08-07 08:34:24 on dist-test-slave-h3ft"
server_key: "c8233856d7b15886d0b4144245f41dbf"
server_key_iv: "547a3df4ab1f7016d1b68e300278b4cc"
server_key_version: "encryptionkey@0"
I20260807 08:34:24.598671 11991 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:34:24.601826 12013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:24.603500 11991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 08:34:24.603644 11991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "829561e191174809a66563f24e24a3d2"
format_stamp: "Formatted at 2026-08-07 08:34:24 on dist-test-slave-h3ft"
server_key: "c8233856d7b15886d0b4144245f41dbf"
server_key_iv: "547a3df4ab1f7016d1b68e300278b4cc"
server_key_version: "encryptionkey@0"
I20260807 08:34:24.603771 11991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:24.620306 11991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:24.624433 11991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:24.624696 11991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:24.626583 11991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:24.626637 11991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:24.626711 11991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:24.626744 11991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:34:24 dist-test-slave-h3ft krb5kdc[11440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091664, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:24.640576 11991 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:40941
I20260807 08:34:24.640663 12169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:40941 every 8 connection(s)
I20260807 08:34:24.641824 11991 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:34:24.643980 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.627818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:38181 (local address 127.9.174.62:35827)
0807 08:34:24.628020 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:24.628025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:24.629356 (+ 1331us) server_negotiation.cc:407] Connection header received
0807 08:34:24.630885 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:24.630891 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:24.631008 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:24.631128 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:24.632900 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.633984 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.635717 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.636028 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.636164 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.639566 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:24.639570 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:24.639572 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:24.639605 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:24.641392 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:24.642156 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:24.642162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:24.642164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:24.642290 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:24.642719 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:24.642722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:24.642725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:24.642984 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:24.643140 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:24.643683 (+ 543us) server_negotiation.cc:299] Negotiation successful
0807 08:34:24.643825 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 08:34:24.645156 12025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.628473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:35827 (local address 127.9.174.3:38181)
0807 08:34:24.629146 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:24.629204 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:24.630544 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:24.631588 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:24.631596 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:24.632019 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:24.632721 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.632740 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.634158 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.634164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.635510 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.635560 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.636236 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.636241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.636454 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.637261 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:24.637314 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:24.639348 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:24.641546 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:24.641553 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:24.641570 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:24.642044 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:24.642427 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:24.642431 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:24.642433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:24.642617 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:24.643132 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:24.643138 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:24.643416 (+ 278us) client_negotiation.cc:769] Sending connection context
0807 08:34:24.643607 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 08:34:24.643937 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":158,"threads_started":1}
I20260807 08:34:24.646116 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 11991
I20260807 08:34:24.646278 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:34:24.646605 9912 external_mini_cluster.cc:1428] Setting key e209127cfd9b72acfa9e3e686fde3795
I20260807 08:34:24.646903 12171 heartbeater.cc:347] Connected to a master server at 127.9.174.62:35827
I20260807 08:34:24.647138 12171 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:24.648002 12171 heartbeater.cc:511] Master 127.9.174.62:35827 requested a full tablet report, sending...
I20260807 08:34:24.649360 11502 ts_manager.cc:194] Registered new tserver with Master: 829561e191174809a66563f24e24a3d2 (127.9.174.3:40941)
I20260807 08:34:24.650151 11502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:38181
I20260807 08:34:24.658192 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.650515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:45035 (local address 127.9.174.62:35827)
0807 08:34:24.650662 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:24.650665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:24.650677 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:34:24.650710 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:24.650712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:24.650750 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:24.650845 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:24.651445 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.652009 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.652917 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.653153 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.653267 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.654734 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:24.654743 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:24.654746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:24.654772 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:24.656318 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:24.656802 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:24.656807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:24.656808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:24.656902 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:24.657216 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:24.657220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:24.657222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:24.657436 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:24.657558 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:24.657855 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 08:34:24.657991 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 08:34:24.658207 12180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.650043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:35827 (local address 127.9.174.3:45035)
0807 08:34:24.650391 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:24.650405 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:24.650494 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:24.650869 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:24.650873 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:24.651189 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:24.651319 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.651327 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.652151 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.652154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.652783 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.652792 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.653278 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.653281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.653397 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.654124 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:24.654137 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:24.654563 (+ 426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:24.656453 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:24.656457 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:24.656460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:24.656694 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:24.657011 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:24.657014 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:24.657017 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:24.657117 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:24.657557 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:24.657560 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:24.657674 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 08:34:24.657792 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 08:34:24.657985 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":149,"threads_started":1}
I20260807 08:34:24.661917 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:34:24 dist-test-slave-h3ft krb5kdc[11440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091664, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:34:24 dist-test-slave-h3ft krb5kdc[11440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091664, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:24.695313 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.685248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39116 (local address 127.9.174.62:35827)
0807 08:34:24.685416 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:24.685420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:24.685438 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:34:24.685497 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:24.685500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:24.685552 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:24.685631 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:24.686482 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.687168 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.688039 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.688290 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.688404 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.691432 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:24.691436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:24.691438 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:24.691475 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:24.693221 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:24.693718 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:24.693721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:24.693723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:24.693817 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:24.694187 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:24.694191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:24.694192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:24.694458 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:24.694589 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:24.694867 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 08:34:24.695142 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 08:34:24.699514 11502 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:34:24.703521 11502 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:34:24.723057 12201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.714849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:40941 (local address 127.0.0.1:32996)
0807 08:34:24.716230 (+ 1381us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:24.716243 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:24.716327 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:24.717287 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:24.717290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:24.717311 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:24.717645 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.717654 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.719679 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.719683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.720974 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.720985 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.722324 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.722328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.722532 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.722555 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:34:24.722668 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 08:34:24.722826 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1318,"mutex_wait_us":317,"thread_start_us":170,"threads_started":1}
I20260807 08:34:24.723285 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.715078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37282 (local address 127.9.174.2:33079)
0807 08:34:24.715817 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:24.715834 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:24.716649 (+ 815us) server_negotiation.cc:407] Connection header received
0807 08:34:24.716861 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:24.716866 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:24.717051 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:24.717171 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:24.717745 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.719083 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.720918 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.721870 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.721962 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.722203 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:24.722830 (+ 627us) server_negotiation.cc:299] Negotiation successful
0807 08:34:24.722954 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":181,"threads_started":1}
I20260807 08:34:24.723695 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.714633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:33079 (local address 127.0.0.1:37282)
0807 08:34:24.716541 (+ 1908us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:24.716560 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:24.716718 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:24.717368 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:24.717381 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:24.717397 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:24.717590 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.717598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.719250 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.719253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.720766 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.720777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.722310 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.722315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.722499 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.722555 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 08:34:24.722676 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 08:34:24.723564 (+ 888us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1839,"mutex_wait_us":1763,"thread_start_us":118,"threads_started":1}
I20260807 08:34:24.723840 12197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.714633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:36385 (local address 127.0.0.1:45340)
0807 08:34:24.716051 (+ 1418us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:24.716117 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:24.716289 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:24.717258 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:24.717266 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:24.717290 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:24.717590 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.717598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.719233 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.719238 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.721089 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.721098 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.722335 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.722338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.723448 (+ 1110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.723480 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 08:34:24.723557 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 08:34:24.723646 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1201,"mutex_wait_us":663,"thread_start_us":131,"threads_started":1}
I20260807 08:34:24.724251 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.716098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32996 (local address 127.9.174.3:40941)
0807 08:34:24.716644 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:24.716656 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:24.716682 (+ 26us) server_negotiation.cc:407] Connection header received
0807 08:34:24.716769 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:24.716774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:24.716969 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:24.717122 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:24.717805 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.719538 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.721130 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.721992 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.722098 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.722261 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:24.723962 (+ 1701us) server_negotiation.cc:299] Negotiation successful
0807 08:34:24.724083 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":143,"threads_started":1}
I20260807 08:34:24.727772 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.715203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45340 (local address 127.9.174.1:36385)
0807 08:34:24.716070 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:24.716077 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:24.716447 (+ 370us) server_negotiation.cc:407] Connection header received
0807 08:34:24.716560 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:24.716569 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:24.716846 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:24.717013 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:24.717744 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.719083 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.721235 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.722038 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.722136 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.722283 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:24.723662 (+ 1379us) server_negotiation.cc:299] Negotiation successful
0807 08:34:24.727613 (+ 3951us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":86,"threads_started":1}
I20260807 08:34:24.727563 11885 tablet_service.cc:1467] Processing CreateTablet for tablet e98fe8e6336f4878bc90d8fb802c23bd (DEFAULT_TABLE table=test-table [id=b43d71336b574d53b12405bd14f1fa04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:24.727567 12077 tablet_service.cc:1467] Processing CreateTablet for tablet e98fe8e6336f4878bc90d8fb802c23bd (DEFAULT_TABLE table=test-table [id=b43d71336b574d53b12405bd14f1fa04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:24.729131 12077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e98fe8e6336f4878bc90d8fb802c23bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:24.731308 11694 tablet_service.cc:1467] Processing CreateTablet for tablet e98fe8e6336f4878bc90d8fb802c23bd (DEFAULT_TABLE table=test-table [id=b43d71336b574d53b12405bd14f1fa04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:24.732348 11885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e98fe8e6336f4878bc90d8fb802c23bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:24.732782 11694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e98fe8e6336f4878bc90d8fb802c23bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:24.738771 12203 tablet_bootstrap.cc:492] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2: Bootstrap starting.
I20260807 08:34:24.741158 12204 tablet_bootstrap.cc:492] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464: Bootstrap starting.
I20260807 08:34:24.741637 12203 tablet_bootstrap.cc:654] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:24.742332 12205 tablet_bootstrap.cc:492] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9: Bootstrap starting.
I20260807 08:34:24.742874 12203 log.cc:824] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:24.743855 12204 tablet_bootstrap.cc:654] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:24.744727 12205 tablet_bootstrap.cc:654] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:24.744867 12204 log.cc:824] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:24.745678 12205 log.cc:824] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:24.746672 12203 tablet_bootstrap.cc:492] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2: No bootstrap required, opened a new log
I20260807 08:34:24.746954 12203 ts_tablet_manager.cc:1397] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 08:34:24.747151 12204 tablet_bootstrap.cc:492] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464: No bootstrap required, opened a new log
I20260807 08:34:24.747370 12204 ts_tablet_manager.cc:1397] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260807 08:34:24.747886 12205 tablet_bootstrap.cc:492] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9: No bootstrap required, opened a new log
I20260807 08:34:24.748189 12205 ts_tablet_manager.cc:1397] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260807 08:34:24.751969 12205 raft_consensus.cc:348] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:24.752045 12203 raft_consensus.cc:348] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:24.752333 12203 raft_consensus.cc:374] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:24.752334 12205 raft_consensus.cc:374] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:24.752399 12203 raft_consensus.cc:729] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 829561e191174809a66563f24e24a3d2, State: Initialized, Role: FOLLOWER
I20260807 08:34:24.752411 12205 raft_consensus.cc:729] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a811d7866acb431983af7856dbc1b1a9, State: Initialized, Role: FOLLOWER
I20260807 08:34:24.752456 12204 raft_consensus.cc:348] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:24.752740 12204 raft_consensus.cc:374] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:24.752813 12204 raft_consensus.cc:729] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50dcad7d4e944899830bf985c9f0d464, State: Initialized, Role: FOLLOWER
I20260807 08:34:24.753062 12203 consensus_queue.cc:260] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:24.753068 12205 consensus_queue.cc:260] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:24.754259 12171 heartbeater.cc:503] Master 127.9.174.62:35827 was elected leader, sending a full tablet report...
I20260807 08:34:24.754432 12205 ts_tablet_manager.cc:1428] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260807 08:34:24.754443 12203 ts_tablet_manager.cc:1428] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 08:34:24.753190 12204 consensus_queue.cc:260] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:24.754823 11978 heartbeater.cc:503] Master 127.9.174.62:35827 was elected leader, sending a full tablet report...
I20260807 08:34:24.755398 11786 heartbeater.cc:503] Master 127.9.174.62:35827 was elected leader, sending a full tablet report...
I20260807 08:34:24.755741 12204 ts_tablet_manager.cc:1428] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 08:34:24.793464 12210 raft_consensus.cc:482] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:34:24.793877 12210 raft_consensus.cc:504] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:24.795795 12210 leader_election.cc:288] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50dcad7d4e944899830bf985c9f0d464 (127.9.174.1:36385), 829561e191174809a66563f24e24a3d2 (127.9.174.3:40941)
I20260807 08:34:24.800654 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.796289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:46021 (local address 127.9.174.3:40941)
0807 08:34:24.796447 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:24.796453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:24.797068 (+ 615us) server_negotiation.cc:407] Connection header received
0807 08:34:24.797202 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:24.797204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:24.797258 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:24.797346 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:24.797694 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.798225 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.799480 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.799926 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.800083 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.800172 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:24.800392 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 08:34:24.800480 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 08:34:24.800673 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.796116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:40941 (local address 127.9.174.2:46021)
0807 08:34:24.796934 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:24.796946 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:24.797086 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:24.797355 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:24.797359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:24.797377 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:24.797545 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.797551 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.798344 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.798350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.799337 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.799348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.800105 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.800109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.800223 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.800273 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 08:34:24.800360 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 08:34:24.800450 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"mutex_wait_us":386,"thread_start_us":189,"threads_started":1}
I20260807 08:34:24.800809 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.796267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:54489 (local address 127.9.174.1:36385)
0807 08:34:24.796451 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:24.796455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:24.796472 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:34:24.796540 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:24.796543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:24.796598 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:24.796704 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:24.797098 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.797692 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.799557 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.800072 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.800215 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.800313 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:24.800552 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 08:34:24.800652 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 08:34:24.800841 11831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.796114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:36385 (local address 127.9.174.2:54489)
0807 08:34:24.796266 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:24.796285 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:24.796398 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:24.796739 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:24.796742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:24.796764 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:24.796935 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.796941 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.797825 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.797829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.799385 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:24.799397 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.800230 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.800235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:24.800406 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.800427 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 08:34:24.800530 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 08:34:24.800617 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":77}
I20260807 08:34:24.801954 12105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e98fe8e6336f4878bc90d8fb802c23bd" candidate_uuid: "a811d7866acb431983af7856dbc1b1a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "829561e191174809a66563f24e24a3d2" is_pre_election: true
I20260807 08:34:24.801954 11718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e98fe8e6336f4878bc90d8fb802c23bd" candidate_uuid: "a811d7866acb431983af7856dbc1b1a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50dcad7d4e944899830bf985c9f0d464" is_pre_election: true
I20260807 08:34:24.802390 12105 raft_consensus.cc:2454] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a811d7866acb431983af7856dbc1b1a9 in term 0.
I20260807 08:34:24.802389 11718 raft_consensus.cc:2454] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a811d7866acb431983af7856dbc1b1a9 in term 0.
I20260807 08:34:24.802983 11826 leader_election.cc:302] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 50dcad7d4e944899830bf985c9f0d464, 829561e191174809a66563f24e24a3d2, a811d7866acb431983af7856dbc1b1a9; no voters:
I20260807 08:34:24.803407 12210 raft_consensus.cc:2781] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:34:24.803519 12210 raft_consensus.cc:482] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:34:24.803587 12210 raft_consensus.cc:3037] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:24.805096 12210 raft_consensus.cc:504] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:24.805601 12210 leader_election.cc:288] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [CANDIDATE]: Term 1 election: Requested vote from peers 50dcad7d4e944899830bf985c9f0d464 (127.9.174.1:36385), 829561e191174809a66563f24e24a3d2 (127.9.174.3:40941)
I20260807 08:34:24.806082 11718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e98fe8e6336f4878bc90d8fb802c23bd" candidate_uuid: "a811d7866acb431983af7856dbc1b1a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50dcad7d4e944899830bf985c9f0d464"
I20260807 08:34:24.806094 12105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e98fe8e6336f4878bc90d8fb802c23bd" candidate_uuid: "a811d7866acb431983af7856dbc1b1a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "829561e191174809a66563f24e24a3d2"
I20260807 08:34:24.806216 11718 raft_consensus.cc:3037] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:24.806267 12105 raft_consensus.cc:3037] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:24.807797 11718 raft_consensus.cc:2454] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a811d7866acb431983af7856dbc1b1a9 in term 1.
I20260807 08:34:24.808374 11821 leader_election.cc:302] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50dcad7d4e944899830bf985c9f0d464, a811d7866acb431983af7856dbc1b1a9; no voters:
I20260807 08:34:24.808542 12105 raft_consensus.cc:2454] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a811d7866acb431983af7856dbc1b1a9 in term 1.
I20260807 08:34:24.808679 12210 raft_consensus.cc:2781] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:24.809007 12210 raft_consensus.cc:686] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [term 1 LEADER]: Becoming Leader. State: Replica: a811d7866acb431983af7856dbc1b1a9, State: Running, Role: LEADER
I20260807 08:34:24.809455 12210 consensus_queue.cc:237] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:24.813848 11501 catalog_manager.cc:5495] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to a811d7866acb431983af7856dbc1b1a9 (127.9.174.2). New cstate: current_term: 1 leader_uuid: "a811d7866acb431983af7856dbc1b1a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } health_report { overall_health: UNKNOWN } } }
W20260807 08:34:24.847730 11788 tablet.cc:2369] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:34:24.859678 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:24.854796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37298 (local address 127.9.174.2:33079)
0807 08:34:24.854994 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:24.855027 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:24.855052 (+ 25us) server_negotiation.cc:407] Connection header received
0807 08:34:24.855435 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:24.855440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:24.855524 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:24.855626 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:24.855960 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.856564 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.857605 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:24.857861 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:24.857966 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:24.858366 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:24.859135 (+ 769us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:24.859248 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:24.859356 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 08:34:24.859437 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 08:34:24.867756 12105 raft_consensus.cc:1261] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2 [term 1 FOLLOWER]: Refusing update from remote peer a811d7866acb431983af7856dbc1b1a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:24.867756 11718 raft_consensus.cc:1261] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464 [term 1 FOLLOWER]: Refusing update from remote peer a811d7866acb431983af7856dbc1b1a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:24.868712 12216 consensus_queue.cc:1035] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:24.869005 12210 consensus_queue.cc:1035] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 08:34:24.869527 11979 tablet.cc:2369] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:34:24.878444 12220 mvcc.cc:205] Tried to move back new op lower bound from 7315831459288350720 to 7315831459063181312. Current Snapshot: MvccSnapshot[applied={T|T < 7315831459288350720}]
I20260807 08:34:24.878786 12221 mvcc.cc:205] Tried to move back new op lower bound from 7315831459288350720 to 7315831459063181312. Current Snapshot: MvccSnapshot[applied={T|T < 7315831459288350720}]
I20260807 08:34:24.879204 12222 mvcc.cc:205] Tried to move back new op lower bound from 7315831459288350720 to 7315831459063181312. Current Snapshot: MvccSnapshot[applied={T|T < 7315831459288350720}]
I20260807 08:34:24.897185 11495 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:24.897541 11495 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:24.901706 11495 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12: Sending DeleteTablet for 3 replicas of tablet e98fe8e6336f4878bc90d8fb802c23bd
I20260807 08:34:24.903047 11885 tablet_service.cc:1514] Processing DeleteTablet for tablet e98fe8e6336f4878bc90d8fb802c23bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:24 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:37282
I20260807 08:34:24.903237 11694 tablet_service.cc:1514] Processing DeleteTablet for tablet e98fe8e6336f4878bc90d8fb802c23bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:24 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:45340
I20260807 08:34:24.903913 12077 tablet_service.cc:1514] Processing DeleteTablet for tablet e98fe8e6336f4878bc90d8fb802c23bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:24 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:32996
I20260807 08:34:24.904253 12231 tablet_replica.cc:330] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9: stopping tablet replica
I20260807 08:34:24.904327 11495 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 08:34:24.904757 12231 raft_consensus.cc:2229] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [term 1 LEADER]: Raft consensus shutting down.
W20260807 08:34:24.904891 11495 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:34:24.905378 12231 raft_consensus.cc:2258] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:24.905949 12233 tablet_replica.cc:330] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2: stopping tablet replica
I20260807 08:34:24.906363 12233 raft_consensus.cc:2229] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:34:24.906785 12233 raft_consensus.cc:2258] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:24.907953 12231 ts_tablet_manager.cc:1905] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:24.908667 12233 ts_tablet_manager.cc:1905] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:24.911697 11694 tablet_service.cc:1467] Processing CreateTablet for tablet 6935558899434c3495ddb9b8f3bd2951 (DEFAULT_TABLE table=test-table [id=98c2096c7f1b489c8f395d69db5ef4b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:24.911814 11885 tablet_service.cc:1467] Processing CreateTablet for tablet 6935558899434c3495ddb9b8f3bd2951 (DEFAULT_TABLE table=test-table [id=98c2096c7f1b489c8f395d69db5ef4b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:24.912106 11885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6935558899434c3495ddb9b8f3bd2951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:24.912117 11694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6935558899434c3495ddb9b8f3bd2951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:24.912642 12077 tablet_service.cc:1467] Processing CreateTablet for tablet 6935558899434c3495ddb9b8f3bd2951 (DEFAULT_TABLE table=test-table [id=98c2096c7f1b489c8f395d69db5ef4b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:24.912914 12077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6935558899434c3495ddb9b8f3bd2951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:24.915442 12205 tablet_bootstrap.cc:492] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9: Bootstrap starting.
I20260807 08:34:24.916080 12204 tablet_bootstrap.cc:492] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464: Bootstrap starting.
I20260807 08:34:24.917567 12233 ts_tablet_manager.cc:1918] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:34:24.917708 12233 log.cc:1196] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/e98fe8e6336f4878bc90d8fb802c23bd
I20260807 08:34:24.917894 12205 tablet_bootstrap.cc:654] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:24.918136 12233 ts_tablet_manager.cc:1939] T e98fe8e6336f4878bc90d8fb802c23bd P 829561e191174809a66563f24e24a3d2: Deleting consensus metadata
I20260807 08:34:24.918591 12204 tablet_bootstrap.cc:654] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:24.919188 12232 tablet_replica.cc:330] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464: stopping tablet replica
I20260807 08:34:24.919229 12203 tablet_bootstrap.cc:492] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2: Bootstrap starting.
I20260807 08:34:24.919682 12232 raft_consensus.cc:2229] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:34:24.919981 11482 catalog_manager.cc:4841] TS 829561e191174809a66563f24e24a3d2 (127.9.174.3:40941): tablet e98fe8e6336f4878bc90d8fb802c23bd (table test-table [id=b43d71336b574d53b12405bd14f1fa04]) successfully deleted
I20260807 08:34:24.920110 12232 raft_consensus.cc:2258] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:24.921509 12203 tablet_bootstrap.cc:654] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:24.921535 12204 tablet_bootstrap.cc:492] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464: No bootstrap required, opened a new log
I20260807 08:34:24.921629 12204 ts_tablet_manager.cc:1397] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260807 08:34:24.922791 12204 raft_consensus.cc:348] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:24.922957 12204 raft_consensus.cc:374] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:24.922994 12204 raft_consensus.cc:729] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50dcad7d4e944899830bf985c9f0d464, State: Initialized, Role: FOLLOWER
I20260807 08:34:24.923275 12204 consensus_queue.cc:260] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:24.923565 12204 ts_tablet_manager.cc:1428] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:34:24.925412 12232 ts_tablet_manager.cc:1905] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:24.927199 12231 ts_tablet_manager.cc:1918] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:34:24.927249 12205 tablet_bootstrap.cc:492] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9: No bootstrap required, opened a new log
I20260807 08:34:24.927511 12205 ts_tablet_manager.cc:1397] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.003s
I20260807 08:34:24.928642 12203 tablet_bootstrap.cc:492] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2: No bootstrap required, opened a new log
I20260807 08:34:24.928783 12203 ts_tablet_manager.cc:1397] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260807 08:34:24.928686 12205 raft_consensus.cc:348] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:24.929288 12205 raft_consensus.cc:374] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:24.929414 12205 raft_consensus.cc:729] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a811d7866acb431983af7856dbc1b1a9, State: Initialized, Role: FOLLOWER
I20260807 08:34:24.929708 12205 consensus_queue.cc:260] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:24.929766 12203 raft_consensus.cc:348] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:24.929939 12203 raft_consensus.cc:374] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:24.929996 12203 raft_consensus.cc:729] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 829561e191174809a66563f24e24a3d2, State: Initialized, Role: FOLLOWER
I20260807 08:34:24.930168 12203 consensus_queue.cc:260] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:24.931524 12232 ts_tablet_manager.cc:1918] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:34:24.931666 12232 log.cc:1196] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/wal/wals/e98fe8e6336f4878bc90d8fb802c23bd
I20260807 08:34:24.931926 12205 ts_tablet_manager.cc:1428] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260807 08:34:24.932068 12232 ts_tablet_manager.cc:1939] T e98fe8e6336f4878bc90d8fb802c23bd P 50dcad7d4e944899830bf985c9f0d464: Deleting consensus metadata
I20260807 08:34:24.932253 12231 log.cc:1196] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/e98fe8e6336f4878bc90d8fb802c23bd
I20260807 08:34:24.932847 12231 ts_tablet_manager.cc:1939] T e98fe8e6336f4878bc90d8fb802c23bd P a811d7866acb431983af7856dbc1b1a9: Deleting consensus metadata
I20260807 08:34:24.933981 11478 catalog_manager.cc:4841] TS 50dcad7d4e944899830bf985c9f0d464 (127.9.174.1:36385): tablet e98fe8e6336f4878bc90d8fb802c23bd (table test-table [id=b43d71336b574d53b12405bd14f1fa04]) successfully deleted
I20260807 08:34:24.934486 12203 ts_tablet_manager.cc:1428] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260807 08:34:24.935042 11481 catalog_manager.cc:4841] TS a811d7866acb431983af7856dbc1b1a9 (127.9.174.2:33079): tablet e98fe8e6336f4878bc90d8fb802c23bd (table test-table [id=b43d71336b574d53b12405bd14f1fa04]) successfully deleted
I20260807 08:34:25.011571 12214 raft_consensus.cc:482] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:34:25.011817 12214 raft_consensus.cc:504] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:25.013491 12214 leader_election.cc:288] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a811d7866acb431983af7856dbc1b1a9 (127.9.174.2:33079), 829561e191174809a66563f24e24a3d2 (127.9.174.3:40941)
I20260807 08:34:25.020640 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:25.014419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:53195 (local address 127.9.174.3:40941)
0807 08:34:25.014599 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:25.014604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:25.014659 (+ 55us) server_negotiation.cc:407] Connection header received
0807 08:34:25.014745 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:25.014749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:25.014813 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:25.014920 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:25.015657 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.016204 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.018955 (+ 2751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.019710 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.019874 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:25.019979 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:25.020359 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 08:34:25.020468 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 08:34:25.020816 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:25.013887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:40941 (local address 127.9.174.1:53195)
0807 08:34:25.014391 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:25.014407 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:25.014582 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:25.014974 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:25.014978 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:25.015031 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:25.015476 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:25.015487 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.016356 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.016361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:25.018718 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:25.018736 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.019906 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.019909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:25.020192 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:25.020219 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 08:34:25.020330 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 08:34:25.020446 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":184,"spinlock_wait_cycles":36864,"thread_start_us":102,"threads_started":1}
I20260807 08:34:25.020932 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:25.013936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:33337 (local address 127.9.174.2:33079)
0807 08:34:25.014136 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:25.014141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:25.014454 (+ 313us) server_negotiation.cc:407] Connection header received
0807 08:34:25.014743 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:25.014749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:25.014816 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:25.014861 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:25.015534 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.016340 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.019159 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.019760 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.019897 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:25.019979 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:25.020649 (+ 670us) server_negotiation.cc:299] Negotiation successful
0807 08:34:25.020765 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 08:34:25.021888 12239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:25.013808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:33079 (local address 127.9.174.1:33337)
0807 08:34:25.014321 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:25.014342 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:25.014582 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:25.015092 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:25.015095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:25.015109 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:25.015303 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:25.015312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.016504 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.016507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:25.018718 (+ 2211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:25.018736 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.020255 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.020261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:25.020444 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:25.020478 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 08:34:25.021324 (+ 846us) client_negotiation.cc:240] Negotiation successful
0807 08:34:25.021432 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":250,"spinlock_wait_cycles":20864,"thread_start_us":167,"threads_started":1}
I20260807 08:34:25.022171 12105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6935558899434c3495ddb9b8f3bd2951" candidate_uuid: "50dcad7d4e944899830bf985c9f0d464" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "829561e191174809a66563f24e24a3d2" is_pre_election: true
I20260807 08:34:25.022414 12105 raft_consensus.cc:2454] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50dcad7d4e944899830bf985c9f0d464 in term 0.
I20260807 08:34:25.022562 11918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6935558899434c3495ddb9b8f3bd2951" candidate_uuid: "50dcad7d4e944899830bf985c9f0d464" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a811d7866acb431983af7856dbc1b1a9" is_pre_election: true
I20260807 08:34:25.022887 11918 raft_consensus.cc:2454] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50dcad7d4e944899830bf985c9f0d464 in term 0.
I20260807 08:34:25.023089 11631 leader_election.cc:302] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50dcad7d4e944899830bf985c9f0d464, 829561e191174809a66563f24e24a3d2; no voters:
I20260807 08:34:25.023787 12214 raft_consensus.cc:2781] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:34:25.023890 12214 raft_consensus.cc:482] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:34:25.023944 12214 raft_consensus.cc:3037] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:25.026226 12214 raft_consensus.cc:504] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:25.026845 12214 leader_election.cc:288] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [CANDIDATE]: Term 1 election: Requested vote from peers a811d7866acb431983af7856dbc1b1a9 (127.9.174.2:33079), 829561e191174809a66563f24e24a3d2 (127.9.174.3:40941)
I20260807 08:34:25.027493 12105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6935558899434c3495ddb9b8f3bd2951" candidate_uuid: "50dcad7d4e944899830bf985c9f0d464" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "829561e191174809a66563f24e24a3d2"
I20260807 08:34:25.027487 11918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6935558899434c3495ddb9b8f3bd2951" candidate_uuid: "50dcad7d4e944899830bf985c9f0d464" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a811d7866acb431983af7856dbc1b1a9"
I20260807 08:34:25.027690 11918 raft_consensus.cc:3037] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:25.027691 12105 raft_consensus.cc:3037] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:25.029362 12105 raft_consensus.cc:2454] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50dcad7d4e944899830bf985c9f0d464 in term 1.
I20260807 08:34:25.029963 11631 leader_election.cc:302] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50dcad7d4e944899830bf985c9f0d464, 829561e191174809a66563f24e24a3d2; no voters:
I20260807 08:34:25.030252 12214 raft_consensus.cc:2781] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:25.030508 11918 raft_consensus.cc:2454] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50dcad7d4e944899830bf985c9f0d464 in term 1.
I20260807 08:34:25.030587 12214 raft_consensus.cc:686] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [term 1 LEADER]: Becoming Leader. State: Replica: 50dcad7d4e944899830bf985c9f0d464, State: Running, Role: LEADER
I20260807 08:34:25.031309 12214 consensus_queue.cc:237] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:25.035076 11502 catalog_manager.cc:5495] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50dcad7d4e944899830bf985c9f0d464 (127.9.174.1). New cstate: current_term: 1 leader_uuid: "50dcad7d4e944899830bf985c9f0d464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } health_report { overall_health: UNKNOWN } } }
Aug 07 08:34:25 dist-test-slave-h3ft krb5kdc[11440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091663, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:25.089603 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:25.078123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39132 (local address 127.9.174.62:35827)
0807 08:34:25.078305 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:25.078310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:25.078784 (+ 474us) server_negotiation.cc:407] Connection header received
0807 08:34:25.078983 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:25.078986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:25.079071 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:25.079174 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:25.080101 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.080695 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.081538 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.081767 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.081880 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:25.085300 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:25.085304 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:25.085306 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:25.085333 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:25.087448 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:25.088063 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:25.088069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:25.088071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:25.088163 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:25.088476 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:25.088480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:25.088482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:25.088721 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:25.088838 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:25.089190 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 08:34:25.089364 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":13,"server-negotiator.queue_time_us":69}
I20260807 08:34:25.089605 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:25.078058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:35827 (local address 127.0.0.1:39132)
0807 08:34:25.078655 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:25.078670 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:25.078868 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:25.079202 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:25.079205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:25.079845 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:25.079967 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:25.079976 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.080812 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.080819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:25.081403 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:25.081411 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.081895 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.081899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:25.082007 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:25.082855 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:25.082869 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:25.085085 (+ 2216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:25.087602 (+ 2517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:25.087622 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:25.087640 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:25.087942 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:25.088261 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:25.088266 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:25.088268 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:25.088362 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:25.088852 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:25.088863 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:25.089003 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 08:34:25.089137 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 08:34:25.089446 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":132,"thread_start_us":274,"threads_started":1}
I20260807 08:34:25.097874 11502 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:39132:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 08:34:25.099206 11502 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:34:25.106467 11694 tablet_service.cc:1467] Processing CreateTablet for tablet cdf81a9c99934e3eb64778ce508bb237 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5569431fd29942aca6e4480c16f6f1f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:34:25.107106 11694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdf81a9c99934e3eb64778ce508bb237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:25.106945 11885 tablet_service.cc:1467] Processing CreateTablet for tablet cdf81a9c99934e3eb64778ce508bb237 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5569431fd29942aca6e4480c16f6f1f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:34:25.107499 11885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdf81a9c99934e3eb64778ce508bb237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:25.112257 12077 tablet_service.cc:1467] Processing CreateTablet for tablet cdf81a9c99934e3eb64778ce508bb237 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5569431fd29942aca6e4480c16f6f1f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:34:25.112876 12077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdf81a9c99934e3eb64778ce508bb237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:25.113336 12204 tablet_bootstrap.cc:492] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464: Bootstrap starting.
I20260807 08:34:25.114880 12204 tablet_bootstrap.cc:654] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:25.117540 12203 tablet_bootstrap.cc:492] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2: Bootstrap starting.
I20260807 08:34:25.119799 12205 tablet_bootstrap.cc:492] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9: Bootstrap starting.
I20260807 08:34:25.119916 12203 tablet_bootstrap.cc:654] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:25.120323 12204 tablet_bootstrap.cc:492] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464: No bootstrap required, opened a new log
I20260807 08:34:25.120436 12204 ts_tablet_manager.cc:1397] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260807 08:34:25.121417 12204 raft_consensus.cc:348] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:25.121614 12204 raft_consensus.cc:374] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:25.121654 12204 raft_consensus.cc:729] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50dcad7d4e944899830bf985c9f0d464, State: Initialized, Role: FOLLOWER
I20260807 08:34:25.121935 12205 tablet_bootstrap.cc:654] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:25.121862 12204 consensus_queue.cc:260] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:25.122448 12204 ts_tablet_manager.cc:1428] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:34:25.123389 12214 tablet_replica.cc:439] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } } }
I20260807 08:34:25.123668 12214 tablet_replica.cc:442] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:34:25.123728 12203 tablet_bootstrap.cc:492] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2: No bootstrap required, opened a new log
I20260807 08:34:25.123850 12203 ts_tablet_manager.cc:1397] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 08:34:25.124961 12205 tablet_bootstrap.cc:492] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9: No bootstrap required, opened a new log
I20260807 08:34:25.124856 12203 raft_consensus.cc:348] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:25.125100 12205 ts_tablet_manager.cc:1397] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260807 08:34:25.125097 12203 raft_consensus.cc:374] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:25.125144 12203 raft_consensus.cc:729] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 829561e191174809a66563f24e24a3d2, State: Initialized, Role: FOLLOWER
I20260807 08:34:25.125353 12203 consensus_queue.cc:260] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:25.125685 12203 ts_tablet_manager.cc:1428] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 08:34:25.125721 12205 raft_consensus.cc:348] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:25.125851 12205 raft_consensus.cc:374] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:25.125885 12205 raft_consensus.cc:729] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a811d7866acb431983af7856dbc1b1a9, State: Initialized, Role: FOLLOWER
I20260807 08:34:25.126019 12205 consensus_queue.cc:260] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:25.126348 12205 ts_tablet_manager.cc:1428] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260807 08:34:25.126550 12213 tablet_replica.cc:439] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } } }
I20260807 08:34:25.126768 12213 tablet_replica.cc:442] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:34:25.126890 12225 tablet_replica.cc:439] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } } }
I20260807 08:34:25.127158 12225 tablet_replica.cc:442] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9: This TxnStatusTablet replica's current role is: FOLLOWER
W20260807 08:34:25.144196 12172 tablet.cc:2369] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:34:25.230077 12214 raft_consensus.cc:482] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:34:25.230268 12214 raft_consensus.cc:504] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:25.230960 12214 leader_election.cc:288] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a811d7866acb431983af7856dbc1b1a9 (127.9.174.2:33079), 829561e191174809a66563f24e24a3d2 (127.9.174.3:40941)
I20260807 08:34:25.231542 11918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdf81a9c99934e3eb64778ce508bb237" candidate_uuid: "50dcad7d4e944899830bf985c9f0d464" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a811d7866acb431983af7856dbc1b1a9" is_pre_election: true
I20260807 08:34:25.231626 12105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdf81a9c99934e3eb64778ce508bb237" candidate_uuid: "50dcad7d4e944899830bf985c9f0d464" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "829561e191174809a66563f24e24a3d2" is_pre_election: true
I20260807 08:34:25.231761 11918 raft_consensus.cc:2454] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50dcad7d4e944899830bf985c9f0d464 in term 0.
I20260807 08:34:25.231864 12105 raft_consensus.cc:2454] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50dcad7d4e944899830bf985c9f0d464 in term 0.
I20260807 08:34:25.232342 11630 leader_election.cc:302] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50dcad7d4e944899830bf985c9f0d464, a811d7866acb431983af7856dbc1b1a9; no voters:
I20260807 08:34:25.232590 12214 raft_consensus.cc:2781] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:34:25.232687 12214 raft_consensus.cc:482] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:34:25.232738 12214 raft_consensus.cc:3037] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:25.234083 12214 raft_consensus.cc:504] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:25.234575 12214 leader_election.cc:288] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [CANDIDATE]: Term 1 election: Requested vote from peers a811d7866acb431983af7856dbc1b1a9 (127.9.174.2:33079), 829561e191174809a66563f24e24a3d2 (127.9.174.3:40941)
I20260807 08:34:25.235148 11918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdf81a9c99934e3eb64778ce508bb237" candidate_uuid: "50dcad7d4e944899830bf985c9f0d464" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a811d7866acb431983af7856dbc1b1a9"
I20260807 08:34:25.235327 11918 raft_consensus.cc:3037] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:25.235335 12105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdf81a9c99934e3eb64778ce508bb237" candidate_uuid: "50dcad7d4e944899830bf985c9f0d464" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "829561e191174809a66563f24e24a3d2"
I20260807 08:34:25.235571 12105 raft_consensus.cc:3037] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:25.236658 11918 raft_consensus.cc:2454] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50dcad7d4e944899830bf985c9f0d464 in term 1.
I20260807 08:34:25.236893 12105 raft_consensus.cc:2454] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50dcad7d4e944899830bf985c9f0d464 in term 1.
I20260807 08:34:25.237110 11630 leader_election.cc:302] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50dcad7d4e944899830bf985c9f0d464, a811d7866acb431983af7856dbc1b1a9; no voters:
I20260807 08:34:25.237361 12214 raft_consensus.cc:2781] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:25.237458 12214 raft_consensus.cc:686] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [term 1 LEADER]: Becoming Leader. State: Replica: 50dcad7d4e944899830bf985c9f0d464, State: Running, Role: LEADER
I20260807 08:34:25.237658 12214 consensus_queue.cc:237] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } }
I20260807 08:34:25.239112 12243 tablet_replica.cc:439] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464: TxnStatusTablet state changed. Reason: New leader 50dcad7d4e944899830bf985c9f0d464. Latest consensus state: current_term: 1 leader_uuid: "50dcad7d4e944899830bf985c9f0d464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } } }
I20260807 08:34:25.239262 12243 tablet_replica.cc:442] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:34:25.239704 12260 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 08:34:25.239912 12260 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 08:34:25.240085 11495 catalog_manager.cc:5495] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50dcad7d4e944899830bf985c9f0d464 (127.9.174.1). New cstate: current_term: 1 leader_uuid: "50dcad7d4e944899830bf985c9f0d464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } health_report { overall_health: UNKNOWN } } }
I20260807 08:34:25.266446 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:25.261860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45346 (local address 127.9.174.1:36385)
0807 08:34:25.262076 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:25.262081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:25.262100 (+ 19us) server_negotiation.cc:407] Connection header received
0807 08:34:25.262249 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:25.262253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:25.262381 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:25.262503 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:25.262840 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.263509 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.264469 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.264721 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.264842 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:25.265209 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:25.265890 (+ 681us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:25.266034 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:25.266130 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 08:34:25.266207 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 08:34:25.266408 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:25.261682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:36385 (local address 127.0.0.1:45346)
0807 08:34:25.261870 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:25.261891 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:25.262069 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:25.262539 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:25.262542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:25.262552 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:34:25.262717 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:25.262724 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.263634 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.263638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:25.264327 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:25.264336 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.264848 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.264852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:25.264965 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:25.265028 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:34:25.265999 (+ 971us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:34:25.266022 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:34:25.266105 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 08:34:25.266233 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 08:34:25.270702 12105 raft_consensus.cc:1261] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2 [term 1 FOLLOWER]: Refusing update from remote peer 50dcad7d4e944899830bf985c9f0d464: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:25.270955 11918 raft_consensus.cc:1261] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9 [term 1 FOLLOWER]: Refusing update from remote peer 50dcad7d4e944899830bf985c9f0d464: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:25.271582 12243 consensus_queue.cc:1035] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [LEADER]: Connected to new peer: Peer: permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:25.271936 12214 consensus_queue.cc:1035] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:25.274284 12225 tablet_replica.cc:439] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9: TxnStatusTablet state changed. Reason: New leader 50dcad7d4e944899830bf985c9f0d464. Latest consensus state: current_term: 1 leader_uuid: "50dcad7d4e944899830bf985c9f0d464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } } }
I20260807 08:34:25.274435 12225 tablet_replica.cc:442] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:34:25.274957 12213 tablet_replica.cc:439] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2: TxnStatusTablet state changed. Reason: New leader 50dcad7d4e944899830bf985c9f0d464. Latest consensus state: current_term: 1 leader_uuid: "50dcad7d4e944899830bf985c9f0d464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } } }
I20260807 08:34:25.275137 12213 tablet_replica.cc:442] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:34:25.277930 12243 tablet_replica.cc:439] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50dcad7d4e944899830bf985c9f0d464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } } }
I20260807 08:34:25.278115 12243 tablet_replica.cc:442] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:34:25.278277 12243 tablet_replica.cc:439] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50dcad7d4e944899830bf985c9f0d464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } } }
I20260807 08:34:25.278383 12243 tablet_replica.cc:442] T cdf81a9c99934e3eb64778ce508bb237 P 50dcad7d4e944899830bf985c9f0d464: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:34:25.281209 12213 tablet_replica.cc:439] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50dcad7d4e944899830bf985c9f0d464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } } }
I20260807 08:34:25.281396 12213 tablet_replica.cc:442] T cdf81a9c99934e3eb64778ce508bb237 P 829561e191174809a66563f24e24a3d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:34:25.281308 12225 tablet_replica.cc:439] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50dcad7d4e944899830bf985c9f0d464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50dcad7d4e944899830bf985c9f0d464" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36385 } } peers { permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 } } peers { permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 } } }
I20260807 08:34:25.281453 12225 tablet_replica.cc:442] T cdf81a9c99934e3eb64778ce508bb237 P a811d7866acb431983af7856dbc1b1a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:34:25.290122 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:25.286368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45356 (local address 127.9.174.1:36385)
0807 08:34:25.286524 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:25.286528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:25.286544 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:34:25.286643 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:25.286646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:25.286711 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:25.286798 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:25.287109 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.287926 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.288888 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.289164 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.289284 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:25.289582 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:25.289681 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:25.289765 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:25.289863 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 08:34:25.289927 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 08:34:25.305825 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:25.301239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:35253 (local address 127.9.174.1:36385)
0807 08:34:25.301386 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:25.301392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:25.302025 (+ 633us) server_negotiation.cc:407] Connection header received
0807 08:34:25.302230 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:25.302233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:25.302285 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:25.302367 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:25.302863 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.303490 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.304571 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.304791 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.304911 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:25.305255 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:25.305346 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:25.305426 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:25.305538 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 08:34:25.305601 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":44}
I20260807 08:34:25.305833 12271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:25.301172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:36385 (local address 127.9.174.1:35253)
0807 08:34:25.301853 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:25.301867 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:25.302121 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:25.302376 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:25.302379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:25.302452 (+ 73us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:34:25.302747 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:25.302754 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.303602 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.303607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:25.304442 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:25.304454 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.304949 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.304955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:25.305048 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:25.305103 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:34:25.305429 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:34:25.305437 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 08:34:25.305527 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 08:34:25.305617 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":228,"threads_started":1}
I20260807 08:34:25.316977 12105 raft_consensus.cc:1261] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2 [term 1 FOLLOWER]: Refusing update from remote peer 50dcad7d4e944899830bf985c9f0d464: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:25.317035 11918 raft_consensus.cc:1261] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9 [term 1 FOLLOWER]: Refusing update from remote peer 50dcad7d4e944899830bf985c9f0d464: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:25.317515 12266 consensus_queue.cc:1035] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a811d7866acb431983af7856dbc1b1a9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:25.317775 12214 consensus_queue.cc:1035] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [LEADER]: Connected to new peer: Peer: permanent_uuid: "829561e191174809a66563f24e24a3d2" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:25.382292 11501 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:25.382551 11501 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:25.386075 11501 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1cfc7bd65b2a44b19144d543f6041d12: Sending DeleteTablet for 3 replicas of tablet 6935558899434c3495ddb9b8f3bd2951
I20260807 08:34:25.386965 11885 tablet_service.cc:1514] Processing DeleteTablet for tablet 6935558899434c3495ddb9b8f3bd2951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:25 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:37282
I20260807 08:34:25.386983 11693 tablet_service.cc:1514] Processing DeleteTablet for tablet 6935558899434c3495ddb9b8f3bd2951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:25 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:45340
I20260807 08:34:25.387286 12231 tablet_replica.cc:330] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9: stopping tablet replica
I20260807 08:34:25.387293 12232 tablet_replica.cc:330] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464: stopping tablet replica
I20260807 08:34:25.387457 12232 raft_consensus.cc:2229] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:34:25.387457 12231 raft_consensus.cc:2229] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:34:25.387630 12231 raft_consensus.cc:2258] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:25.387915 12232 raft_consensus.cc:2258] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:25.388733 12231 ts_tablet_manager.cc:1905] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:25.388873 12232 ts_tablet_manager.cc:1905] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:25.392999 12231 ts_tablet_manager.cc:1918] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:34:25.393111 12231 log.cc:1196] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/6935558899434c3495ddb9b8f3bd2951
I20260807 08:34:25.393435 12231 ts_tablet_manager.cc:1939] T 6935558899434c3495ddb9b8f3bd2951 P a811d7866acb431983af7856dbc1b1a9: Deleting consensus metadata
I20260807 08:34:25.393431 12232 ts_tablet_manager.cc:1918] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:34:25.393679 12232 log.cc:1196] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-0/wal/wals/6935558899434c3495ddb9b8f3bd2951
I20260807 08:34:25.394017 12232 ts_tablet_manager.cc:1939] T 6935558899434c3495ddb9b8f3bd2951 P 50dcad7d4e944899830bf985c9f0d464: Deleting consensus metadata
I20260807 08:34:25.394584 11481 catalog_manager.cc:4841] TS a811d7866acb431983af7856dbc1b1a9 (127.9.174.2:33079): tablet 6935558899434c3495ddb9b8f3bd2951 (table test-table [id=98c2096c7f1b489c8f395d69db5ef4b0]) successfully deleted
I20260807 08:34:25.395290 11478 catalog_manager.cc:4841] TS 50dcad7d4e944899830bf985c9f0d464 (127.9.174.1:36385): tablet 6935558899434c3495ddb9b8f3bd2951 (table test-table [id=98c2096c7f1b489c8f395d69db5ef4b0]) successfully deleted
I20260807 08:34:25.395735 12077 tablet_service.cc:1514] Processing DeleteTablet for tablet 6935558899434c3495ddb9b8f3bd2951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:25 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:32996
I20260807 08:34:25.396034 12233 tablet_replica.cc:330] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2: stopping tablet replica
I20260807 08:34:25.396195 12233 raft_consensus.cc:2229] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:34:25.396385 12233 raft_consensus.cc:2258] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:25.397535 12233 ts_tablet_manager.cc:1905] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:25.401885 12233 ts_tablet_manager.cc:1918] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:34:25.402026 12233 log.cc:1196] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/6935558899434c3495ddb9b8f3bd2951
I20260807 08:34:25.402383 12233 ts_tablet_manager.cc:1939] T 6935558899434c3495ddb9b8f3bd2951 P 829561e191174809a66563f24e24a3d2: Deleting consensus metadata
I20260807 08:34:25.403460 11482 catalog_manager.cc:4841] TS 829561e191174809a66563f24e24a3d2 (127.9.174.3:40941): tablet 6935558899434c3495ddb9b8f3bd2951 (table test-table [id=98c2096c7f1b489c8f395d69db5ef4b0]) successfully deleted
Aug 07 08:34:25 dist-test-slave-h3ft krb5kdc[11440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091664, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.174.1@KRBTEST.COM
I20260807 08:34:25.413134 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:25.398600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45372 (local address 127.9.174.1:36385)
0807 08:34:25.398754 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:25.398760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:25.399355 (+ 595us) server_negotiation.cc:407] Connection header received
0807 08:34:25.399661 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:25.399665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:25.399729 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:25.399788 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:25.401027 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.401849 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.403348 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.403665 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.403757 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:25.408200 (+ 4443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:25.408217 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:25.408248 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:25.408288 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:25.410554 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:25.411226 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:25.411233 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:25.411235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:25.411378 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:25.411720 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:25.411723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:25.411724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:25.412070 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:25.412257 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:25.412721 (+ 464us) server_negotiation.cc:299] Negotiation successful
0807 08:34:25.412859 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 08:34:25.413800 11774 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45372
I20260807 08:34:25.426822 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:25.416452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39136 (local address 127.9.174.62:35827)
0807 08:34:25.416623 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:25.416628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:25.416645 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:34:25.416839 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:25.416842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:25.416932 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:25.417011 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:25.417994 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.418649 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.419975 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:25.420252 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:25.420411 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:25.422735 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:25.422739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:25.422742 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:25.422782 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:25.424658 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:25.425221 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:25.425224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:25.425225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:25.425325 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:25.425641 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:25.425644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:25.425645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:25.425885 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:25.426003 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:25.426355 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 08:34:25.426565 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260807 08:34:25.427434 11501 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39136
I20260807 08:34:25.429311 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 11608
I20260807 08:34:25.451612 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 11801
I20260807 08:34:25.473969 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 11991
I20260807 08:34:25.493451 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 11455
2026-08-07T08:34:25Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3475 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:34:25 dist-test-slave-h3ft krb5kdc[12284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:34:25 dist-test-slave-h3ft krb5kdc[12284](info): set up 2 sockets
Aug 07 08:34:25 dist-test-slave-h3ft krb5kdc[12284](info): commencing operation
krb5kdc: starting...
W20260807 08:34:26.796078 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.004s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:34:26 dist-test-slave-h3ft krb5kdc[12284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:34:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:34:26.954883 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:37957
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:37957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:27.096099 12302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:27.096501 12302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:27.096549 12302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:27.101147 12302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:27.101256 12302 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 08:34:27.101276 12302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:27.101296 12302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:27.101313 12302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:27.101330 12302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:27.107872 12302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:37957
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:37957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:27.109499 12302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:27.110971 12302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:27.121960 12312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:27.121966 12314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:27.121956 12311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:27.123059 12302 server_base.cc:1034] running on GCE node
I20260807 08:34:27.123899 12302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:27.125502 12302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:27.126703 12302 hybrid_clock.cc:648] HybridClock initialized: now 1786091667126668 us; error 46 us; skew 500 ppm
Aug 07 08:34:27 dist-test-slave-h3ft krb5kdc[12284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091667, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:27.134421 12302 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:34:27.135986 12302 webserver.cc:460] Webserver started at http://127.9.174.62:37887/ using document root <none> and password file <none>
I20260807 08:34:27.136773 12302 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:27.136879 12302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:27.137216 12302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:27.140019 12302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "45ae3ee37adf4f30851ad671b00a0f08"
format_stamp: "Formatted at 2026-08-07 08:34:27 on dist-test-slave-h3ft"
server_key: "297113ab3c1114b67184a827b431f28b"
server_key_iv: "e4e7d9ab072c8d0fc60bc0b98b22668a"
server_key_version: "encryptionkey@0"
I20260807 08:34:27.140717 12302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "45ae3ee37adf4f30851ad671b00a0f08"
format_stamp: "Formatted at 2026-08-07 08:34:27 on dist-test-slave-h3ft"
server_key: "297113ab3c1114b67184a827b431f28b"
server_key_iv: "e4e7d9ab072c8d0fc60bc0b98b22668a"
server_key_version: "encryptionkey@0"
I20260807 08:34:27.145762 12302 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:34:27.148972 12325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:27.150708 12302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:34:27.150906 12302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "45ae3ee37adf4f30851ad671b00a0f08"
format_stamp: "Formatted at 2026-08-07 08:34:27 on dist-test-slave-h3ft"
server_key: "297113ab3c1114b67184a827b431f28b"
server_key_iv: "e4e7d9ab072c8d0fc60bc0b98b22668a"
server_key_version: "encryptionkey@0"
I20260807 08:34:27.151144 12302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:27.171607 12302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:27.176045 12302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:27.176324 12302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:27.186949 12302 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:37957
I20260807 08:34:27.186986 12408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:37957 every 8 connection(s)
I20260807 08:34:27.188390 12302 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:27.192477 12409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:27.196218 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 12302
I20260807 08:34:27.196486 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:34:27.196810 9912 external_mini_cluster.cc:1428] Setting key 035b3981163b3e9c5bae820d9e1bd8a1
I20260807 08:34:27.200891 12409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08: Bootstrap starting.
I20260807 08:34:27.204105 12409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:27.205154 12409 log.cc:824] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:34:27 dist-test-slave-h3ft krb5kdc[12284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:27.208668 12409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08: No bootstrap required, opened a new log
I20260807 08:34:27.214573 12409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ae3ee37adf4f30851ad671b00a0f08" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37957 } }
I20260807 08:34:27.214962 12409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:27.215041 12409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ae3ee37adf4f30851ad671b00a0f08, State: Initialized, Role: FOLLOWER
I20260807 08:34:27.215687 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:27.198844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37510 (local address 127.9.174.62:37957)
0807 08:34:27.199521 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:27.199542 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:27.199590 (+ 48us) server_negotiation.cc:407] Connection header received
0807 08:34:27.200386 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:27.200409 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:27.200847 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:27.201341 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:27.202300 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.203608 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.204719 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.205288 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.205484 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:27.209010 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:27.209018 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:27.209043 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:27.209091 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:27.211543 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:27.212130 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:27.212140 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:27.212153 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:27.212347 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:27.212681 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:27.212684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:27.212686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:27.213413 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:27.213647 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:27.213962 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 08:34:27.214371 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":203,"threads_started":1}
I20260807 08:34:27.215670 12409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ae3ee37adf4f30851ad671b00a0f08" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37957 } }
I20260807 08:34:27.215878 12409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:34:27.215950 12409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:34:27.216070 12409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:27.217633 12409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ae3ee37adf4f30851ad671b00a0f08" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37957 } }
I20260807 08:34:27.218091 12409 leader_election.cc:302] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45ae3ee37adf4f30851ad671b00a0f08; no voters:
I20260807 08:34:27.218585 12409 leader_election.cc:288] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:34:27.218760 12415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:27.219079 12415 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [term 1 LEADER]: Becoming Leader. State: Replica: 45ae3ee37adf4f30851ad671b00a0f08, State: Running, Role: LEADER
I20260807 08:34:27.219537 12415 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ae3ee37adf4f30851ad671b00a0f08" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37957 } }
I20260807 08:34:27.220284 12409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:27.221649 12416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45ae3ee37adf4f30851ad671b00a0f08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ae3ee37adf4f30851ad671b00a0f08" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37957 } } }
I20260807 08:34:27.221853 12416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:27.222275 12417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45ae3ee37adf4f30851ad671b00a0f08. Latest consensus state: current_term: 1 leader_uuid: "45ae3ee37adf4f30851ad671b00a0f08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ae3ee37adf4f30851ad671b00a0f08" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37957 } } }
I20260807 08:34:27.222345 12426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:27.222424 12417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:27.227056 12426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:27.234712 12426 catalog_manager.cc:1347] Generated new cluster ID: a5c133b500bf4f04acad0ca622cf1706
I20260807 08:34:27.234826 12426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:27.245976 12426 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:27.247336 12426 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:27.261406 12426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08: Generated new TSK 0
I20260807 08:34:27.262655 12426 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:34:27.306824 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37957
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:27.456986 12444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:27.457391 12444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:27.457453 12444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:27.462387 12444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:27.462504 12444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:27.462556 12444 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 08:34:27.462574 12444 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:34:27.462670 12444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:34:27.469725 12444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.174.62:37957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:27.471426 12444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:27.473129 12444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:27.486708 12452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:27.486824 12456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:27.486709 12453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:27.487468 12444 server_base.cc:1034] running on GCE node
I20260807 08:34:27.488147 12444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:27.489323 12444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:27.490715 12444 hybrid_clock.cc:648] HybridClock initialized: now 1786091667490663 us; error 62 us; skew 500 ppm
Aug 07 08:34:27 dist-test-slave-h3ft krb5kdc[12284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091667, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:27.496035 12444 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:34:27.501901 12444 webserver.cc:460] Webserver started at http://127.9.174.1:40251/ using document root <none> and password file <none>
I20260807 08:34:27.503104 12444 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:27.503269 12444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:27.503666 12444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:27.507905 12444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "3324942fa45b462488e68cd2fb620407"
format_stamp: "Formatted at 2026-08-07 08:34:27 on dist-test-slave-h3ft"
server_key: "1b9d5079f3f3fc8d2f29c9d3532ce46b"
server_key_iv: "8b6e169adcad5e18968df0ffed578a84"
server_key_version: "encryptionkey@0"
I20260807 08:34:27.508937 12444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "3324942fa45b462488e68cd2fb620407"
format_stamp: "Formatted at 2026-08-07 08:34:27 on dist-test-slave-h3ft"
server_key: "1b9d5079f3f3fc8d2f29c9d3532ce46b"
server_key_iv: "8b6e169adcad5e18968df0ffed578a84"
server_key_version: "encryptionkey@0"
I20260807 08:34:27.516649 12444 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260807 08:34:27.521579 12464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:27.523940 12444 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 08:34:27.524143 12444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "3324942fa45b462488e68cd2fb620407"
format_stamp: "Formatted at 2026-08-07 08:34:27 on dist-test-slave-h3ft"
server_key: "1b9d5079f3f3fc8d2f29c9d3532ce46b"
server_key_iv: "8b6e169adcad5e18968df0ffed578a84"
server_key_version: "encryptionkey@0"
I20260807 08:34:27.524335 12444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:27.554507 12444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:27.561203 12444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:27.561537 12444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:27.563875 12444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:27.563941 12444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:27.563992 12444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:27.564042 12444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:34:27 dist-test-slave-h3ft krb5kdc[12284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091667, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:27.578258 12444 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:36079
I20260807 08:34:27.579123 12621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:36079 every 8 connection(s)
I20260807 08:34:27.579618 12444 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:34:27.581936 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:27.565090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:48413 (local address 127.9.174.62:37957)
0807 08:34:27.565269 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:27.565274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:27.566703 (+ 1429us) server_negotiation.cc:407] Connection header received
0807 08:34:27.568156 (+ 1453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:27.568161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:27.568250 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:27.568350 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:27.570021 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.570693 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.572306 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.572649 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.572781 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:27.577049 (+ 4268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:27.577054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:27.577056 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:27.577097 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:27.579059 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:27.579831 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:27.579837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:27.579839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:27.579967 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:27.580427 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:27.580431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:27.580432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:27.580738 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:27.580863 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:27.581517 (+ 654us) server_negotiation.cc:299] Negotiation successful
0807 08:34:27.581711 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 08:34:27.582517 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 12444
I20260807 08:34:27.582672 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:34:27.583096 12478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:27.565817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37957 (local address 127.9.174.1:48413)
0807 08:34:27.566487 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:27.566547 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:27.567831 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:27.568604 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:27.568616 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:27.569138 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:27.569822 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:27.569839 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.570835 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.570839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:27.572130 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:27.572148 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.572858 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.572862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:27.573080 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:27.574038 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:27.574063 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:27.576742 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:27.579264 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:27.579275 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:27.579299 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:27.579680 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:27.580076 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:27.580080 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:27.580082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:27.580310 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:27.580859 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:27.580867 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:27.581230 (+ 363us) client_negotiation.cc:769] Sending connection context
0807 08:34:27.581456 (+ 226us) client_negotiation.cc:240] Negotiation successful
0807 08:34:27.581881 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":172,"threads_started":1}
I20260807 08:34:27.583118 9912 external_mini_cluster.cc:1428] Setting key 31b77a53d9d9d6a70503e3f97906ce41
I20260807 08:34:27.585134 12622 heartbeater.cc:347] Connected to a master server at 127.9.174.62:37957
I20260807 08:34:27.585327 12622 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:27.586099 12622 heartbeater.cc:511] Master 127.9.174.62:37957 requested a full tablet report, sending...
I20260807 08:34:27.588430 12344 ts_manager.cc:194] Registered new tserver with Master: 3324942fa45b462488e68cd2fb620407 (127.9.174.1:36079)
I20260807 08:34:27.590159 12344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:48413
I20260807 08:34:27.596841 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:27.588367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:37385 (local address 127.9.174.62:37957)
0807 08:34:27.588525 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:27.588532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:27.588757 (+ 225us) server_negotiation.cc:407] Connection header received
0807 08:34:27.588938 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:27.588943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:27.589006 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:27.589104 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:27.589723 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.590315 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.591509 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.591750 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.591870 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:27.593314 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:27.593318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:27.593321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:27.593355 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:27.594934 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:27.595448 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:27.595452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:27.595454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:27.595550 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:27.595846 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:27.595850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:27.595852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:27.596074 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:27.596188 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:27.596488 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 08:34:27.596637 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 08:34:27.596848 12638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:27.588256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37957 (local address 127.9.174.1:37385)
0807 08:34:27.588668 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:27.588686 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:27.588812 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:27.589147 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:27.589150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:27.589432 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:27.589585 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:27.589595 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.590471 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.590474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:27.591381 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:27.591391 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.591878 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.591882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:27.591987 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:27.592656 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:27.592677 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:27.593160 (+ 483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:27.595089 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:27.595093 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:27.595095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:27.595332 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:27.595649 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:27.595652 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:27.595655 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:27.595753 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:27.596179 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:27.596182 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:27.596295 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 08:34:27.596405 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 08:34:27.596601 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:34:27.624185 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37957
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:27.775488 12642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:27.775934 12642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:27.776036 12642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:27.780727 12642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:27.780864 12642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:27.780954 12642 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 08:34:27.780997 12642 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:34:27.781072 12642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:34:27.788002 12642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.174.62:37957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:27.789690 12642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:27.791167 12642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:27.804255 12655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:27.804255 12652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:27.804250 12653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:27.804750 12642 server_base.cc:1034] running on GCE node
I20260807 08:34:27.805336 12642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:27.806313 12642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:27.807528 12642 hybrid_clock.cc:648] HybridClock initialized: now 1786091667807487 us; error 63 us; skew 500 ppm
Aug 07 08:34:27 dist-test-slave-h3ft krb5kdc[12284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091667, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:27.811641 12642 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:34:27.813036 12642 webserver.cc:460] Webserver started at http://127.9.174.2:42445/ using document root <none> and password file <none>
I20260807 08:34:27.813812 12642 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:27.813913 12642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:27.814205 12642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:27.817050 12642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "4ceece55bcf6455e92ba5d8d74106772"
format_stamp: "Formatted at 2026-08-07 08:34:27 on dist-test-slave-h3ft"
server_key: "2f178822a73f3b045cca6f150fcd51c8"
server_key_iv: "46e5f2d4b0f85a6cecc8734c7bf17f96"
server_key_version: "encryptionkey@0"
I20260807 08:34:27.817798 12642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "4ceece55bcf6455e92ba5d8d74106772"
format_stamp: "Formatted at 2026-08-07 08:34:27 on dist-test-slave-h3ft"
server_key: "2f178822a73f3b045cca6f150fcd51c8"
server_key_iv: "46e5f2d4b0f85a6cecc8734c7bf17f96"
server_key_version: "encryptionkey@0"
I20260807 08:34:27.823177 12642 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 08:34:27.826434 12664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:27.828105 12642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:34:27.828279 12642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "4ceece55bcf6455e92ba5d8d74106772"
format_stamp: "Formatted at 2026-08-07 08:34:27 on dist-test-slave-h3ft"
server_key: "2f178822a73f3b045cca6f150fcd51c8"
server_key_iv: "46e5f2d4b0f85a6cecc8734c7bf17f96"
server_key_version: "encryptionkey@0"
I20260807 08:34:27.828434 12642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:27.847441 12642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:27.852463 12642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:27.853006 12642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:27.855504 12642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:27.855613 12642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:27.855700 12642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:27.855746 12642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:34:27 dist-test-slave-h3ft krb5kdc[12284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091667, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:27.870527 12642 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:44191
I20260807 08:34:27.870636 12824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:44191 every 8 connection(s)
I20260807 08:34:27.872057 12642 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:34:27.875422 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:27.857032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:50315 (local address 127.9.174.62:37957)
0807 08:34:27.857224 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:27.857229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:27.859501 (+ 2272us) server_negotiation.cc:407] Connection header received
0807 08:34:27.861274 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:27.861280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:27.861386 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:27.861515 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:27.863377 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.864079 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.865858 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.866190 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.866375 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:27.870411 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:27.870416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:27.870420 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:27.870480 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:27.872348 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:27.873168 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:27.873173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:27.873176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:27.873294 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:27.873745 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:27.873749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:27.873750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:27.874089 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:27.874203 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:27.874899 (+ 696us) server_negotiation.cc:299] Negotiation successful
0807 08:34:27.875239 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 08:34:27.875782 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 12642
I20260807 08:34:27.875916 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:34:27.876294 9912 external_mini_cluster.cc:1428] Setting key 053da2088d15112e76e0453f25e77be2
I20260807 08:34:27.876760 12679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:27.857857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37957 (local address 127.9.174.2:50315)
0807 08:34:27.859064 (+ 1207us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:27.859214 (+ 150us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:27.860909 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:27.861789 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:27.861799 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:27.862326 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:27.863141 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:27.863166 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.864273 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.864277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:27.865684 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:27.865699 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.866449 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.866455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:27.866717 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:27.867661 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:27.867694 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:27.870077 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:27.872507 (+ 2430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:27.872514 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:27.872533 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:27.872990 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:27.873402 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:27.873406 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:27.873409 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:27.873594 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:27.874207 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:27.874214 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:27.874547 (+ 333us) client_negotiation.cc:769] Sending connection context
0807 08:34:27.874742 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 08:34:27.875239 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":199,"threads_started":1}
I20260807 08:34:27.878537 12825 heartbeater.cc:347] Connected to a master server at 127.9.174.62:37957
I20260807 08:34:27.878763 12825 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:27.879689 12825 heartbeater.cc:511] Master 127.9.174.62:37957 requested a full tablet report, sending...
I20260807 08:34:27.881351 12344 ts_manager.cc:194] Registered new tserver with Master: 4ceece55bcf6455e92ba5d8d74106772 (127.9.174.2:44191)
I20260807 08:34:27.882313 12344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:50315
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
I20260807 08:34:27.891600 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:27.881726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:38953 (local address 127.9.174.62:37957)
0807 08:34:27.881880 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:27.881885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:27.882180 (+ 295us) server_negotiation.cc:407] Connection header received
0807 08:34:27.882433 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:27.882436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:27.882487 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:27.882598 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:27.883407 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.884196 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.885619 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.885913 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.886066 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:27.887754 (+ 1688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:27.887758 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:27.887760 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:27.887789 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:27.889406 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:27.890007 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:27.890010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:27.890011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:27.890100 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:27.890424 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:27.890426 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:27.890427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:27.890715 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:27.890798 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:27.891282 (+ 484us) server_negotiation.cc:299] Negotiation successful
0807 08:34:27.891424 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 08:34:27.891676 12835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:27.881656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37957 (local address 127.9.174.2:38953)
0807 08:34:27.882047 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:27.882085 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:27.882284 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:27.882632 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:27.882637 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:27.882976 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:27.883231 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:27.883245 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.884378 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.884385 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:27.885472 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:27.885491 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:27.886051 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:27.886065 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:27.886213 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:27.886889 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:27.886913 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:27.887586 (+ 673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:27.889565 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:27.889572 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:27.889576 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:27.889902 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:27.890199 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:27.890204 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:27.890207 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:27.890325 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:27.890826 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:27.890832 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:27.890988 (+ 156us) client_negotiation.cc:769] Sending connection context
0807 08:34:27.891174 (+ 186us) client_negotiation.cc:240] Negotiation successful
0807 08:34:27.891404 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":145,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:34:27.919580 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37957
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:28.066794 12837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:28.067274 12837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:28.067384 12837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:28.072333 12837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:28.072465 12837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:28.072517 12837 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 08:34:28.072533 12837 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:34:28.072571 12837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:34:28.079613 12837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.174.62:37957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:28.081269 12837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:28.082737 12837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:28.095983 12847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:28.095992 12849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:28.095992 12846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:28.096508 12837 server_base.cc:1034] running on GCE node
I20260807 08:34:28.097183 12837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:28.098171 12837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:28.099398 12837 hybrid_clock.cc:648] HybridClock initialized: now 1786091668099369 us; error 38 us; skew 500 ppm
Aug 07 08:34:28 dist-test-slave-h3ft krb5kdc[12284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091668, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:28.103238 12837 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:34:28.104539 12837 webserver.cc:460] Webserver started at http://127.9.174.3:40227/ using document root <none> and password file <none>
I20260807 08:34:28.105302 12837 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:28.105377 12837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:28.105610 12837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:28.108771 12837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "6f574837f91546cdb3002f5d9c3ddcc8"
format_stamp: "Formatted at 2026-08-07 08:34:28 on dist-test-slave-h3ft"
server_key: "864214318dafa57d31ee58a251f38656"
server_key_iv: "ce815d96999040387601e1db8cc070fb"
server_key_version: "encryptionkey@0"
I20260807 08:34:28.109436 12837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "6f574837f91546cdb3002f5d9c3ddcc8"
format_stamp: "Formatted at 2026-08-07 08:34:28 on dist-test-slave-h3ft"
server_key: "864214318dafa57d31ee58a251f38656"
server_key_iv: "ce815d96999040387601e1db8cc070fb"
server_key_version: "encryptionkey@0"
I20260807 08:34:28.114212 12837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 08:34:28.117296 12859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:28.118824 12837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:34:28.118954 12837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "6f574837f91546cdb3002f5d9c3ddcc8"
format_stamp: "Formatted at 2026-08-07 08:34:28 on dist-test-slave-h3ft"
server_key: "864214318dafa57d31ee58a251f38656"
server_key_iv: "ce815d96999040387601e1db8cc070fb"
server_key_version: "encryptionkey@0"
I20260807 08:34:28.119145 12837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:28.133467 12837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:28.137810 12837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:28.138096 12837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:28.140138 12837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:28.140219 12837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:28.140316 12837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:28.140363 12837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:34:28 dist-test-slave-h3ft krb5kdc[12284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091668, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:28.154901 12837 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:37171
I20260807 08:34:28.154959 13022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:37171 every 8 connection(s)
I20260807 08:34:28.156409 12837 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:34:28.158401 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.141298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:55959 (local address 127.9.174.62:37957)
0807 08:34:28.141472 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:28.141477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:28.142924 (+ 1447us) server_negotiation.cc:407] Connection header received
0807 08:34:28.144447 (+ 1523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:28.144454 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:28.144567 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:28.144686 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:28.146512 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.147200 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.148713 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.148967 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.149100 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.153136 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:28.153140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:28.153143 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:28.153183 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:28.155445 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:28.156239 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:28.156243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:28.156245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:28.156380 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:28.156846 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:28.156850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:28.156852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:28.157171 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:28.157291 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:28.158078 (+ 787us) server_negotiation.cc:299] Negotiation successful
0807 08:34:28.158239 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 08:34:28.159601 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.142048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37957 (local address 127.9.174.3:55959)
0807 08:34:28.142715 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:28.142770 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:28.144044 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:28.145016 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:28.145030 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:28.145527 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:28.146271 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.146293 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.147363 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.147367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.148548 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.148562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.149189 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.149193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.149418 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.150345 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:28.150386 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:28.152785 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:28.155610 (+ 2825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:28.155616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:28.155642 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:28.156109 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:28.156511 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:28.156515 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:28.156518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:28.156732 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:28.157362 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:28.157369 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:28.157772 (+ 403us) client_negotiation.cc:769] Sending connection context
0807 08:34:28.157984 (+ 212us) client_negotiation.cc:240] Negotiation successful
0807 08:34:28.158332 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":183,"threads_started":1}
I20260807 08:34:28.160570 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 12837
I20260807 08:34:28.160719 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:34:28.161159 9912 external_mini_cluster.cc:1428] Setting key ac683e1ba7858f571bc472887bd9ac7c
I20260807 08:34:28.161679 13023 heartbeater.cc:347] Connected to a master server at 127.9.174.62:37957
I20260807 08:34:28.161880 13023 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:28.162828 13023 heartbeater.cc:511] Master 127.9.174.62:37957 requested a full tablet report, sending...
I20260807 08:34:28.164289 12347 ts_manager.cc:194] Registered new tserver with Master: 6f574837f91546cdb3002f5d9c3ddcc8 (127.9.174.3:37171)
I20260807 08:34:28.165242 12347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:55959
I20260807 08:34:28.172675 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.164806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:53117 (local address 127.9.174.62:37957)
0807 08:34:28.164961 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:28.164966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:28.165207 (+ 241us) server_negotiation.cc:407] Connection header received
0807 08:34:28.165322 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:28.165329 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:28.165388 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:28.165486 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:28.166100 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.166899 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.167812 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.168124 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.168217 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.169472 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:28.169482 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:28.169484 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:28.169512 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:28.171183 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:28.171589 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:28.171592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:28.171594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:28.171674 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:28.171903 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:28.171905 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:28.171906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:28.172106 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:28.172187 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:28.172400 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 08:34:28.172515 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 08:34:28.172688 13031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.164692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37957 (local address 127.9.174.3:53117)
0807 08:34:28.165112 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:28.165127 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:28.165239 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:28.165506 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:28.165509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:28.165825 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:28.165977 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.165984 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.167093 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.167097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.167693 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.167701 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.168237 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.168240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.168312 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.168905 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:28.168917 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:28.169330 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:28.171285 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:28.171288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:28.171290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:28.171505 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:28.171751 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:28.171754 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:28.171755 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:28.171831 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:28.172184 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:28.172186 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:28.172279 (+ 93us) client_negotiation.cc:769] Sending connection context
0807 08:34:28.172362 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 08:34:28.172502 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":155,"threads_started":1}
I20260807 08:34:28.176465 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:34:28 dist-test-slave-h3ft krb5kdc[12284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:34:28 dist-test-slave-h3ft krb5kdc[12284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:28.210852 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.201128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37520 (local address 127.9.174.62:37957)
0807 08:34:28.201342 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:28.201353 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:28.201480 (+ 127us) server_negotiation.cc:407] Connection header received
0807 08:34:28.201668 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:28.201671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:28.201745 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:28.201867 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:28.202763 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.203507 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.204471 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.204707 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.204810 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.207714 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:28.207718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:28.207720 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:28.207755 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:28.209263 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:28.209695 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:28.209697 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:28.209699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:28.209785 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:28.210020 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:28.210022 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:28.210023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:28.210231 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:28.210342 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:28.210571 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 08:34:28.210689 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 08:34:28.214798 12347 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37520:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:34:28.218765 12347 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:34:28.238868 13052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.232602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:37171 (local address 127.0.0.1:56592)
0807 08:34:28.232910 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:28.232924 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:28.233013 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:28.234589 (+ 1576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:28.234592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:28.234611 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:28.234743 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.234754 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.236068 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.236072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.237189 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.237198 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.238299 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.238302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.238385 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.238402 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 08:34:28.238493 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 08:34:28.238669 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":8,"thread_start_us":139,"threads_started":1}
I20260807 08:34:28.238897 13048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.231388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57980 (local address 127.9.174.2:44191)
0807 08:34:28.231909 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:28.231929 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:28.232324 (+ 395us) server_negotiation.cc:407] Connection header received
0807 08:34:28.232714 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:28.232721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:28.233057 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:28.233268 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:28.233902 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.235273 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.236818 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.237796 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.237910 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.238123 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:28.238641 (+ 518us) server_negotiation.cc:299] Negotiation successful
0807 08:34:28.238745 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":152,"threads_started":1}
I20260807 08:34:28.238871 13053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.233239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56592 (local address 127.9.174.3:37171)
0807 08:34:28.233988 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:28.234004 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:28.234029 (+ 25us) server_negotiation.cc:407] Connection header received
0807 08:34:28.234132 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:28.234190 (+ 58us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:28.234418 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:28.234612 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:28.234872 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.235939 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.237324 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.238182 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.238300 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.238486 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:28.238551 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 08:34:28.238673 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":168,"threads_started":1}
I20260807 08:34:28.238858 13050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.231887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36072 (local address 127.9.174.1:36079)
0807 08:34:28.232228 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:28.232235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:28.232870 (+ 635us) server_negotiation.cc:407] Connection header received
0807 08:34:28.232988 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:28.232993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:28.233331 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:28.233521 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:28.234098 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.235358 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.237092 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.237966 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.238057 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.238226 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:28.238615 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 08:34:28.238720 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":108,"threads_started":1}
I20260807 08:34:28.239429 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.231054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:36079 (local address 127.0.0.1:36072)
0807 08:34:28.232586 (+ 1532us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:28.232605 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:28.232751 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:28.233801 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:28.233807 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:28.233829 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:28.233934 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.233940 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.235468 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.235472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.236939 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.236950 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.238332 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.238336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.238437 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.238470 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 08:34:28.239181 (+ 711us) client_negotiation.cc:240] Negotiation successful
0807 08:34:28.239282 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1249,"mutex_wait_us":57,"thread_start_us":141,"threads_started":1}
I20260807 08:34:28.239949 13045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.230967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:44191 (local address 127.0.0.1:57980)
0807 08:34:28.232148 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:28.232223 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:28.232478 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:28.233517 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:28.233524 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:28.233549 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:28.233739 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.233748 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.235403 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.235407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.236662 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.236672 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.238184 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.238186 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.238318 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.238371 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 08:34:28.238496 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 08:34:28.239844 (+ 1348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":935,"mutex_wait_us":1251,"thread_start_us":135,"threads_started":1}
I20260807 08:34:28.243261 12734 tablet_service.cc:1467] Processing CreateTablet for tablet 1a26eb88d9c444b89228deff48ac7f70 (DEFAULT_TABLE table=test-table [id=ff36c652951f42a2b89ec66ee5703fc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:28.243261 12932 tablet_service.cc:1467] Processing CreateTablet for tablet 1a26eb88d9c444b89228deff48ac7f70 (DEFAULT_TABLE table=test-table [id=ff36c652951f42a2b89ec66ee5703fc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:28.243974 12536 tablet_service.cc:1467] Processing CreateTablet for tablet 1a26eb88d9c444b89228deff48ac7f70 (DEFAULT_TABLE table=test-table [id=ff36c652951f42a2b89ec66ee5703fc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:28.245016 12932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a26eb88d9c444b89228deff48ac7f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:28.245013 12734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a26eb88d9c444b89228deff48ac7f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:28.245229 12536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a26eb88d9c444b89228deff48ac7f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:28.254798 13054 tablet_bootstrap.cc:492] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407: Bootstrap starting.
I20260807 08:34:28.256906 13055 tablet_bootstrap.cc:492] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8: Bootstrap starting.
I20260807 08:34:28.257120 13056 tablet_bootstrap.cc:492] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772: Bootstrap starting.
I20260807 08:34:28.258569 13054 tablet_bootstrap.cc:654] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:28.259652 13056 tablet_bootstrap.cc:654] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:28.259698 13055 tablet_bootstrap.cc:654] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:28.259790 13054 log.cc:824] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:28.260596 13056 log.cc:824] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:28.260605 13055 log.cc:824] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:28.263198 13054 tablet_bootstrap.cc:492] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407: No bootstrap required, opened a new log
I20260807 08:34:28.263247 13056 tablet_bootstrap.cc:492] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772: No bootstrap required, opened a new log
I20260807 08:34:28.263243 13055 tablet_bootstrap.cc:492] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8: No bootstrap required, opened a new log
I20260807 08:34:28.263489 13055 ts_tablet_manager.cc:1397] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 08:34:28.263492 13054 ts_tablet_manager.cc:1397] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 08:34:28.263491 13056 ts_tablet_manager.cc:1397] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260807 08:34:28.267321 13055 raft_consensus.cc:348] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.267700 13055 raft_consensus.cc:374] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:28.267763 13055 raft_consensus.cc:729] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f574837f91546cdb3002f5d9c3ddcc8, State: Initialized, Role: FOLLOWER
I20260807 08:34:28.268193 13056 raft_consensus.cc:348] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.268364 13055 consensus_queue.cc:260] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.268528 13056 raft_consensus.cc:374] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:28.268596 13056 raft_consensus.cc:729] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ceece55bcf6455e92ba5d8d74106772, State: Initialized, Role: FOLLOWER
I20260807 08:34:28.268851 13054 raft_consensus.cc:348] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.269184 13054 raft_consensus.cc:374] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:28.269134 13056 consensus_queue.cc:260] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.269239 13054 raft_consensus.cc:729] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3324942fa45b462488e68cd2fb620407, State: Initialized, Role: FOLLOWER
I20260807 08:34:28.269415 13023 heartbeater.cc:503] Master 127.9.174.62:37957 was elected leader, sending a full tablet report...
I20260807 08:34:28.269701 13054 consensus_queue.cc:260] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.269861 13055 ts_tablet_manager.cc:1428] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 08:34:28.270716 12622 heartbeater.cc:503] Master 127.9.174.62:37957 was elected leader, sending a full tablet report...
I20260807 08:34:28.270783 13054 ts_tablet_manager.cc:1428] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 08:34:28.271922 13056 ts_tablet_manager.cc:1428] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772: Time spent starting tablet: real 0.008s user 0.002s sys 0.005s
I20260807 08:34:28.272329 12825 heartbeater.cc:503] Master 127.9.174.62:37957 was elected leader, sending a full tablet report...
I20260807 08:34:28.310940 13060 raft_consensus.cc:482] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:34:28.311357 13060 raft_consensus.cc:504] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.313393 13060 leader_election.cc:288] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ceece55bcf6455e92ba5d8d74106772 (127.9.174.2:44191), 3324942fa45b462488e68cd2fb620407 (127.9.174.1:36079)
I20260807 08:34:28.317981 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.313769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:36079 (local address 127.9.174.3:60181)
0807 08:34:28.313934 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:28.313953 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:28.314083 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:28.314583 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:28.314588 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:28.314613 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:28.314793 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.314801 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.315716 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.315720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.316766 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.316778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.317556 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.317560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.317676 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.317698 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:34:28.317761 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:34:28.317828 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 08:34:28.318050 13050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.314080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:60181 (local address 127.9.174.1:36079)
0807 08:34:28.314274 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:28.314280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:28.314297 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:34:28.314373 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:28.314379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:28.314452 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:28.314540 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:28.314924 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.315588 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.316912 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.317415 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.317544 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.317634 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:28.317802 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 08:34:28.317900 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 08:34:28.318897 13063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.313769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:44191 (local address 127.9.174.3:53823)
0807 08:34:28.314642 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:28.314658 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:28.314786 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:28.315100 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:28.315103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:28.315120 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:28.315739 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.315748 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.316515 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.316519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.317425 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.317434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.318409 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.318413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.318547 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.318575 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 08:34:28.318661 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 08:34:28.318737 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":402,"thread_start_us":237,"threads_started":1}
I20260807 08:34:28.318892 13048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.314083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:53823 (local address 127.9.174.2:44191)
0807 08:34:28.314275 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:28.314279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:28.314752 (+ 473us) server_negotiation.cc:407] Connection header received
0807 08:34:28.314902 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:28.314905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:28.314978 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:28.315071 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:28.315870 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.316397 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.317575 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.318231 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.318348 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.318418 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:28.318704 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 08:34:28.318780 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 08:34:28.318984 12557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a26eb88d9c444b89228deff48ac7f70" candidate_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3324942fa45b462488e68cd2fb620407" is_pre_election: true
I20260807 08:34:28.319494 12557 raft_consensus.cc:2454] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f574837f91546cdb3002f5d9c3ddcc8 in term 0.
I20260807 08:34:28.319911 12761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a26eb88d9c444b89228deff48ac7f70" candidate_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ceece55bcf6455e92ba5d8d74106772" is_pre_election: true
I20260807 08:34:28.320293 12761 raft_consensus.cc:2454] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f574837f91546cdb3002f5d9c3ddcc8 in term 0.
I20260807 08:34:28.320268 12865 leader_election.cc:302] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3324942fa45b462488e68cd2fb620407, 6f574837f91546cdb3002f5d9c3ddcc8; no voters:
I20260807 08:34:28.320658 13060 raft_consensus.cc:2781] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:34:28.320783 13060 raft_consensus.cc:482] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:34:28.320851 13060 raft_consensus.cc:3037] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:28.322547 13060 raft_consensus.cc:504] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.322985 13060 leader_election.cc:288] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [CANDIDATE]: Term 1 election: Requested vote from peers 4ceece55bcf6455e92ba5d8d74106772 (127.9.174.2:44191), 3324942fa45b462488e68cd2fb620407 (127.9.174.1:36079)
I20260807 08:34:28.323520 12761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a26eb88d9c444b89228deff48ac7f70" candidate_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ceece55bcf6455e92ba5d8d74106772"
I20260807 08:34:28.323668 12761 raft_consensus.cc:3037] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:28.323644 12557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a26eb88d9c444b89228deff48ac7f70" candidate_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3324942fa45b462488e68cd2fb620407"
I20260807 08:34:28.323819 12557 raft_consensus.cc:3037] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:28.325156 12761 raft_consensus.cc:2454] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f574837f91546cdb3002f5d9c3ddcc8 in term 1.
I20260807 08:34:28.325232 12557 raft_consensus.cc:2454] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f574837f91546cdb3002f5d9c3ddcc8 in term 1.
I20260807 08:34:28.325609 12862 leader_election.cc:302] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ceece55bcf6455e92ba5d8d74106772, 6f574837f91546cdb3002f5d9c3ddcc8; no voters:
I20260807 08:34:28.325872 13060 raft_consensus.cc:2781] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:28.326161 13060 raft_consensus.cc:686] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 LEADER]: Becoming Leader. State: Replica: 6f574837f91546cdb3002f5d9c3ddcc8, State: Running, Role: LEADER
I20260807 08:34:28.326663 13060 consensus_queue.cc:237] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.330976 12347 catalog_manager.cc:5495] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f574837f91546cdb3002f5d9c3ddcc8 (127.9.174.3). New cstate: current_term: 1 leader_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } health_report { overall_health: HEALTHY } } }
W20260807 08:34:28.332165 12625 tablet.cc:2369] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:34:28.373839 12827 tablet.cc:2369] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 08:34:28 dist-test-slave-h3ft krb5kdc[12284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091667, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:28.396085 13075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.385750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37957 (local address 127.0.0.1:37532)
0807 08:34:28.386207 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:28.386222 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:28.386406 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:28.386684 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:28.386686 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:28.387316 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:28.387469 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.387478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.388342 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.388345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.388985 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.388995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.389521 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.389525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.389596 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.390393 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:28.390408 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:28.392312 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:28.394214 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:28.394226 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:28.394241 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:28.394534 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:28.394830 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:28.394832 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:28.394834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:28.394940 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:28.395450 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:28.395457 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:28.395593 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 08:34:28.395705 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 08:34:28.395872 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":218,"threads_started":1}
I20260807 08:34:28.396205 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.385815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37532 (local address 127.9.174.62:37957)
0807 08:34:28.386002 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:28.386009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:28.386310 (+ 301us) server_negotiation.cc:407] Connection header received
0807 08:34:28.386508 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:28.386510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:28.386561 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:28.386640 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:28.387612 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.388206 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.389133 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.389370 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.389470 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.392503 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:28.392507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:28.392509 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:28.392535 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:28.394070 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:28.394632 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:28.394635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:28.394637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:28.394719 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:28.395098 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:28.395101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:28.395102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:28.395333 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:28.395436 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:28.395826 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 08:34:28.396012 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 08:34:28.402514 12347 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:37532:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 08:34:28.403748 12347 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 08:34:28.407878 13024 tablet.cc:2369] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:34:28.411315 12536 tablet_service.cc:1467] Processing CreateTablet for tablet 248c8a50080648438ff13a73ba0dede5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2da6177a6f1848b698dcd5ffe5ee32b7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:34:28.411315 12932 tablet_service.cc:1467] Processing CreateTablet for tablet 248c8a50080648438ff13a73ba0dede5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2da6177a6f1848b698dcd5ffe5ee32b7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:34:28.411247 12734 tablet_service.cc:1467] Processing CreateTablet for tablet 248c8a50080648438ff13a73ba0dede5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2da6177a6f1848b698dcd5ffe5ee32b7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:34:28.412266 12932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248c8a50080648438ff13a73ba0dede5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:28.412264 12734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248c8a50080648438ff13a73ba0dede5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:28.412264 12536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248c8a50080648438ff13a73ba0dede5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:28.416160 13055 tablet_bootstrap.cc:492] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8: Bootstrap starting.
I20260807 08:34:28.416474 13054 tablet_bootstrap.cc:492] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407: Bootstrap starting.
I20260807 08:34:28.416900 13056 tablet_bootstrap.cc:492] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772: Bootstrap starting.
I20260807 08:34:28.417598 13054 tablet_bootstrap.cc:654] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:28.417632 13055 tablet_bootstrap.cc:654] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:28.418104 13056 tablet_bootstrap.cc:654] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:28.419255 13054 tablet_bootstrap.cc:492] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407: No bootstrap required, opened a new log
I20260807 08:34:28.419363 13054 ts_tablet_manager.cc:1397] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:34:28.419591 13056 tablet_bootstrap.cc:492] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772: No bootstrap required, opened a new log
I20260807 08:34:28.419690 13056 ts_tablet_manager.cc:1397] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260807 08:34:28.419939 13055 tablet_bootstrap.cc:492] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8: No bootstrap required, opened a new log
I20260807 08:34:28.419936 13054 raft_consensus.cc:348] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.420045 13055 ts_tablet_manager.cc:1397] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 08:34:28.420056 13054 raft_consensus.cc:374] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:28.420079 13054 raft_consensus.cc:729] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3324942fa45b462488e68cd2fb620407, State: Initialized, Role: FOLLOWER
I20260807 08:34:28.420231 13054 consensus_queue.cc:260] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.420229 13056 raft_consensus.cc:348] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.420357 13056 raft_consensus.cc:374] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:28.420390 13056 raft_consensus.cc:729] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ceece55bcf6455e92ba5d8d74106772, State: Initialized, Role: FOLLOWER
I20260807 08:34:28.420545 13054 ts_tablet_manager.cc:1428] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:34:28.420544 13056 consensus_queue.cc:260] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.420568 13061 tablet_replica.cc:439] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } } }
I20260807 08:34:28.420603 13055 raft_consensus.cc:348] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.420719 13055 raft_consensus.cc:374] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:28.420746 13061 tablet_replica.cc:442] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:34:28.420760 13055 raft_consensus.cc:729] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f574837f91546cdb3002f5d9c3ddcc8, State: Initialized, Role: FOLLOWER
I20260807 08:34:28.420893 13055 consensus_queue.cc:260] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.421240 13055 ts_tablet_manager.cc:1428] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:34:28.421357 13062 tablet_replica.cc:439] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } } }
I20260807 08:34:28.421541 13062 tablet_replica.cc:442] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:34:28.422137 13060 tablet_replica.cc:439] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } } }
I20260807 08:34:28.422293 13060 tablet_replica.cc:442] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:34:28.422546 13056 ts_tablet_manager.cc:1428] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 08:34:28.522305 13062 raft_consensus.cc:482] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:34:28.522553 13062 raft_consensus.cc:504] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.524288 13062 leader_election.cc:288] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3324942fa45b462488e68cd2fb620407 (127.9.174.1:36079), 6f574837f91546cdb3002f5d9c3ddcc8 (127.9.174.3:37171)
I20260807 08:34:28.529098 13053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.524797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:55405 (local address 127.9.174.3:37171)
0807 08:34:28.524973 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:28.524979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:28.525183 (+ 204us) server_negotiation.cc:407] Connection header received
0807 08:34:28.525394 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:28.525397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:28.525449 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:28.525545 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:28.525889 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.526488 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.527909 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.528413 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.528505 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.528612 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:28.528832 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 08:34:28.528928 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 08:34:28.529119 13050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.524799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:52665 (local address 127.9.174.1:36079)
0807 08:34:28.525036 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:28.525043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:28.525151 (+ 108us) server_negotiation.cc:407] Connection header received
0807 08:34:28.525384 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:28.525388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:28.525442 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:28.525506 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:28.525928 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.526492 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.527918 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.528440 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.528583 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.528664 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:28.528873 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 08:34:28.528988 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260807 08:34:28.529119 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.524603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:36079 (local address 127.9.174.2:52665)
0807 08:34:28.525047 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:28.525059 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:28.525231 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:28.525544 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:28.525548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:28.525579 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:28.525772 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.525778 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.526618 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.526623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.527772 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.527783 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.528611 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.528615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.528726 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.528745 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 08:34:28.528842 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 08:34:28.528925 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":80,"spinlock_wait_cycles":8192,"thread_start_us":123,"threads_started":1}
I20260807 08:34:28.529119 13082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.524596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:37171 (local address 127.9.174.2:55405)
0807 08:34:28.525032 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:28.525051 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:28.525250 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:28.525582 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:28.525585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:28.525602 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:28.525761 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.525767 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.526616 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.526621 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.527767 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.527779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.528577 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.528581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.528676 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.528710 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 08:34:28.528784 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 08:34:28.528899 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":107,"spinlock_wait_cycles":84352,"thread_start_us":155,"threads_started":1}
I20260807 08:34:28.529788 12557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248c8a50080648438ff13a73ba0dede5" candidate_uuid: "4ceece55bcf6455e92ba5d8d74106772" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3324942fa45b462488e68cd2fb620407" is_pre_election: true
I20260807 08:34:28.530016 12557 raft_consensus.cc:2454] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ceece55bcf6455e92ba5d8d74106772 in term 0.
I20260807 08:34:28.530246 12961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248c8a50080648438ff13a73ba0dede5" candidate_uuid: "4ceece55bcf6455e92ba5d8d74106772" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" is_pre_election: true
I20260807 08:34:28.530560 12961 raft_consensus.cc:2454] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ceece55bcf6455e92ba5d8d74106772 in term 0.
I20260807 08:34:28.530587 12672 leader_election.cc:302] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3324942fa45b462488e68cd2fb620407, 4ceece55bcf6455e92ba5d8d74106772; no voters:
I20260807 08:34:28.531024 13062 raft_consensus.cc:2781] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:34:28.531105 13062 raft_consensus.cc:482] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:34:28.531154 13062 raft_consensus.cc:3037] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:28.532614 13062 raft_consensus.cc:504] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.533053 13062 leader_election.cc:288] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [CANDIDATE]: Term 1 election: Requested vote from peers 3324942fa45b462488e68cd2fb620407 (127.9.174.1:36079), 6f574837f91546cdb3002f5d9c3ddcc8 (127.9.174.3:37171)
I20260807 08:34:28.533504 12557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248c8a50080648438ff13a73ba0dede5" candidate_uuid: "4ceece55bcf6455e92ba5d8d74106772" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3324942fa45b462488e68cd2fb620407"
I20260807 08:34:28.533512 12961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248c8a50080648438ff13a73ba0dede5" candidate_uuid: "4ceece55bcf6455e92ba5d8d74106772" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f574837f91546cdb3002f5d9c3ddcc8"
I20260807 08:34:28.533669 12557 raft_consensus.cc:3037] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:28.533675 12961 raft_consensus.cc:3037] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:28.534988 12961 raft_consensus.cc:2454] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ceece55bcf6455e92ba5d8d74106772 in term 1.
I20260807 08:34:28.535092 12557 raft_consensus.cc:2454] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ceece55bcf6455e92ba5d8d74106772 in term 1.
I20260807 08:34:28.535392 12667 leader_election.cc:302] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ceece55bcf6455e92ba5d8d74106772, 6f574837f91546cdb3002f5d9c3ddcc8; no voters:
I20260807 08:34:28.535650 13062 raft_consensus.cc:2781] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:28.535912 13062 raft_consensus.cc:686] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [term 1 LEADER]: Becoming Leader. State: Replica: 4ceece55bcf6455e92ba5d8d74106772, State: Running, Role: LEADER
I20260807 08:34:28.536369 13062 consensus_queue.cc:237] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.539178 13083 tablet_replica.cc:439] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772: TxnStatusTablet state changed. Reason: New leader 4ceece55bcf6455e92ba5d8d74106772. Latest consensus state: current_term: 1 leader_uuid: "4ceece55bcf6455e92ba5d8d74106772" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } } }
I20260807 08:34:28.539337 13083 tablet_replica.cc:442] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:34:28.539676 13086 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 08:34:28.540014 13086 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 08:34:28.541203 12347 catalog_manager.cc:5495] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ceece55bcf6455e92ba5d8d74106772 (127.9.174.2). New cstate: current_term: 1 leader_uuid: "4ceece55bcf6455e92ba5d8d74106772" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } health_report { overall_health: UNKNOWN } } }
I20260807 08:34:28.582460 13048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.578048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.9.174.2:44191)
0807 08:34:28.578219 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:28.578224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:28.578243 (+ 19us) server_negotiation.cc:407] Connection header received
0807 08:34:28.578521 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:28.578524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:28.578597 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:28.578718 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:28.579066 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.579728 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.580684 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.580920 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.581033 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.581412 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:28.581945 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:28.582077 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:28.582195 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 08:34:28.582268 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 08:34:28.582460 13075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.577803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:44191 (local address 127.0.0.1:55034)
0807 08:34:28.578102 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:28.578123 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:28.578277 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:28.578740 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:28.578743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:28.578754 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:34:28.578909 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.578923 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.579851 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.579854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.580540 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.580554 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.581044 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.581047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.581169 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.581227 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:34:28.582060 (+ 833us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:34:28.582079 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 08:34:28.582169 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 08:34:28.582292 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260807 08:34:28.589850 12557 raft_consensus.cc:1261] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407 [term 1 FOLLOWER]: Refusing update from remote peer 4ceece55bcf6455e92ba5d8d74106772: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:28.589849 12961 raft_consensus.cc:1261] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 FOLLOWER]: Refusing update from remote peer 4ceece55bcf6455e92ba5d8d74106772: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:28.590657 13083 consensus_queue.cc:1035] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:28.590906 13062 consensus_queue.cc:1035] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:28.595258 13060 tablet_replica.cc:439] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8: TxnStatusTablet state changed. Reason: New leader 4ceece55bcf6455e92ba5d8d74106772. Latest consensus state: current_term: 1 leader_uuid: "4ceece55bcf6455e92ba5d8d74106772" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } } }
I20260807 08:34:28.595460 13060 tablet_replica.cc:442] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:34:28.596565 13061 tablet_replica.cc:439] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407: TxnStatusTablet state changed. Reason: New leader 4ceece55bcf6455e92ba5d8d74106772. Latest consensus state: current_term: 1 leader_uuid: "4ceece55bcf6455e92ba5d8d74106772" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } } }
I20260807 08:34:28.596729 13061 tablet_replica.cc:442] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:34:28.598922 13062 tablet_replica.cc:439] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ceece55bcf6455e92ba5d8d74106772" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } } }
I20260807 08:34:28.599108 13062 tablet_replica.cc:442] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:34:28.599265 13090 mvcc.cc:205] Tried to move back new op lower bound from 7315831474534359040 to 7315831474328354816. Current Snapshot: MvccSnapshot[applied={T|T < 7315831474534359040}]
I20260807 08:34:28.601152 13060 tablet_replica.cc:439] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ceece55bcf6455e92ba5d8d74106772" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } } }
I20260807 08:34:28.601294 13060 tablet_replica.cc:442] T 248c8a50080648438ff13a73ba0dede5 P 6f574837f91546cdb3002f5d9c3ddcc8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:34:28.601630 13093 mvcc.cc:205] Tried to move back new op lower bound from 7315831474534359040 to 7315831474328354816. Current Snapshot: MvccSnapshot[applied={T|T < 7315831474534359040}]
I20260807 08:34:28.603062 13083 tablet_replica.cc:439] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ceece55bcf6455e92ba5d8d74106772" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } } }
I20260807 08:34:28.603223 13083 tablet_replica.cc:442] T 248c8a50080648438ff13a73ba0dede5 P 4ceece55bcf6455e92ba5d8d74106772: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:34:28.604668 13061 tablet_replica.cc:439] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ceece55bcf6455e92ba5d8d74106772" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } } }
I20260807 08:34:28.604836 13061 tablet_replica.cc:442] T 248c8a50080648438ff13a73ba0dede5 P 3324942fa45b462488e68cd2fb620407: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:34:28.605273 13096 mvcc.cc:205] Tried to move back new op lower bound from 7315831474534359040 to 7315831474328354816. Current Snapshot: MvccSnapshot[applied={T|T < 7315831474534359040}]
I20260807 08:34:28.615134 13053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.610178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38118 (local address 127.9.174.3:37171)
0807 08:34:28.610313 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:28.610317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:28.610332 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:34:28.610507 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:28.610510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:28.610578 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:28.610685 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:28.610961 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.611601 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.612968 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.613221 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.613320 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.613696 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:28.614408 (+ 712us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:28.614522 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:28.614712 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 08:34:28.614796 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:34:28.630837 13048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.626497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:59869 (local address 127.9.174.2:44191)
0807 08:34:28.626661 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:28.626668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:28.626686 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:34:28.626777 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:28.626780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:28.626843 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:28.626940 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:28.627287 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.628164 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.629185 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.629564 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.629677 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.630235 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:28.630348 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:28.630425 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:28.630588 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 08:34:28.630657 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 08:34:28.630888 13031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.626369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:44191 (local address 127.9.174.3:59869)
0807 08:34:28.626549 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:28.626568 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:28.626676 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:28.626984 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:28.626986 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:28.626992 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:34:28.627171 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.627177 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.628283 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.628286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.629058 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.629069 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.629766 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.629771 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.629911 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.629982 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:34:28.630473 (+ 491us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:34:28.630481 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 08:34:28.630555 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 08:34:28.630652 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 08:34:28.641145 12557 raft_consensus.cc:1261] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407 [term 1 FOLLOWER]: Refusing update from remote peer 6f574837f91546cdb3002f5d9c3ddcc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:28.641419 12761 raft_consensus.cc:1261] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Refusing update from remote peer 6f574837f91546cdb3002f5d9c3ddcc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:28.641903 13060 consensus_queue.cc:1035] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:28.642132 13064 consensus_queue.cc:1035] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:28.674871 13053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.669785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:49067 (local address 127.9.174.3:37171)
0807 08:34:28.670116 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:28.670123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:28.670237 (+ 114us) server_negotiation.cc:407] Connection header received
0807 08:34:28.670393 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:28.670398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:28.670499 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:28.670580 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:28.671067 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.671674 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.673160 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.673582 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.673680 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.674290 (+ 610us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:28.674420 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:28.674476 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:28.674640 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 08:34:28.674711 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235}
I20260807 08:34:28.675112 13126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:28.669662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:37171 (local address 127.9.174.2:49067)
0807 08:34:28.670103 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:28.670121 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:28.670265 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:28.670727 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:28.670730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:28.670741 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:34:28.670897 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.670906 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.671810 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.671814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.673009 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:28.673022 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:28.673858 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:28.673864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:28.674024 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:28.674100 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:34:28.674545 (+ 445us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:34:28.674551 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:34:28.674615 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:34:28.674732 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":158,"threads_started":1}
I20260807 08:34:28.714924 12347 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:28.715276 12347 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:28.719365 12347 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08: Sending DeleteTablet for 3 replicas of tablet 1a26eb88d9c444b89228deff48ac7f70
I20260807 08:34:28.721240 12536 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a26eb88d9c444b89228deff48ac7f70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:28 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:36072
I20260807 08:34:28.721609 12932 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a26eb88d9c444b89228deff48ac7f70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:28 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:56592
I20260807 08:34:28.722322 12734 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a26eb88d9c444b89228deff48ac7f70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:28 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:57980
I20260807 08:34:28.723246 13132 tablet_replica.cc:330] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407: stopping tablet replica
I20260807 08:34:28.723613 12347 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37520:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 08:34:28.724066 13132 raft_consensus.cc:2229] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260807 08:34:28.724261 12347 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:34:28.724495 13132 raft_consensus.cc:2258] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:28.725059 13135 tablet_replica.cc:330] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772: stopping tablet replica
I20260807 08:34:28.727113 13132 ts_tablet_manager.cc:1905] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:28.727638 13135 raft_consensus.cc:2229] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:34:28.727941 13135 raft_consensus.cc:2258] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:28.731657 12536 tablet_service.cc:1467] Processing CreateTablet for tablet ea48a2d65fa342b5bb09f20b33b3243e (DEFAULT_TABLE table=test-table [id=a783ad5cf362442eb95f7a5a65c5a83e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:28.732087 12536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea48a2d65fa342b5bb09f20b33b3243e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:28.733255 13135 ts_tablet_manager.cc:1905] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:28.736014 12734 tablet_service.cc:1467] Processing CreateTablet for tablet ea48a2d65fa342b5bb09f20b33b3243e (DEFAULT_TABLE table=test-table [id=a783ad5cf362442eb95f7a5a65c5a83e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:28.736390 12734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea48a2d65fa342b5bb09f20b33b3243e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:28.737763 13132 ts_tablet_manager.cc:1918] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:34:28.739176 13135 ts_tablet_manager.cc:1918] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:34:28.739410 13132 log.cc:1196] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/wal/wals/1a26eb88d9c444b89228deff48ac7f70
I20260807 08:34:28.739817 12932 tablet_service.cc:1467] Processing CreateTablet for tablet ea48a2d65fa342b5bb09f20b33b3243e (DEFAULT_TABLE table=test-table [id=a783ad5cf362442eb95f7a5a65c5a83e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:28.739948 13132 ts_tablet_manager.cc:1939] T 1a26eb88d9c444b89228deff48ac7f70 P 3324942fa45b462488e68cd2fb620407: Deleting consensus metadata
I20260807 08:34:28.740192 12932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea48a2d65fa342b5bb09f20b33b3243e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:28.742074 13135 log.cc:1196] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/1a26eb88d9c444b89228deff48ac7f70
I20260807 08:34:28.742566 13135 ts_tablet_manager.cc:1939] T 1a26eb88d9c444b89228deff48ac7f70 P 4ceece55bcf6455e92ba5d8d74106772: Deleting consensus metadata
I20260807 08:34:28.743209 12329 catalog_manager.cc:4841] TS 3324942fa45b462488e68cd2fb620407 (127.9.174.1:36079): tablet 1a26eb88d9c444b89228deff48ac7f70 (table test-table [id=ff36c652951f42a2b89ec66ee5703fc1]) successfully deleted
I20260807 08:34:28.743672 13056 tablet_bootstrap.cc:492] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772: Bootstrap starting.
I20260807 08:34:28.744877 13056 tablet_bootstrap.cc:654] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:28.744953 12327 catalog_manager.cc:4841] TS 4ceece55bcf6455e92ba5d8d74106772 (127.9.174.2:44191): tablet 1a26eb88d9c444b89228deff48ac7f70 (table test-table [id=ff36c652951f42a2b89ec66ee5703fc1]) successfully deleted
I20260807 08:34:28.746277 13054 tablet_bootstrap.cc:492] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407: Bootstrap starting.
I20260807 08:34:28.746471 13056 tablet_bootstrap.cc:492] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772: No bootstrap required, opened a new log
I20260807 08:34:28.746580 13056 ts_tablet_manager.cc:1397] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260807 08:34:28.747155 13056 raft_consensus.cc:348] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.747319 13056 raft_consensus.cc:374] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:28.747356 13056 raft_consensus.cc:729] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ceece55bcf6455e92ba5d8d74106772, State: Initialized, Role: FOLLOWER
I20260807 08:34:28.747500 13134 tablet_replica.cc:330] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8: stopping tablet replica
I20260807 08:34:28.747515 13056 consensus_queue.cc:260] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.747646 13054 tablet_bootstrap.cc:654] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:28.747826 13056 ts_tablet_manager.cc:1428] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:34:28.748248 13134 raft_consensus.cc:2229] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:34:28.748827 13134 raft_consensus.cc:2258] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:28.748297 13055 tablet_bootstrap.cc:492] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8: Bootstrap starting.
I20260807 08:34:28.750025 13054 tablet_bootstrap.cc:492] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407: No bootstrap required, opened a new log
I20260807 08:34:28.750149 13054 ts_tablet_manager.cc:1397] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 08:34:28.750731 13054 raft_consensus.cc:348] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.750895 13054 raft_consensus.cc:374] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:28.750936 13055 tablet_bootstrap.cc:654] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:28.750949 13054 raft_consensus.cc:729] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3324942fa45b462488e68cd2fb620407, State: Initialized, Role: FOLLOWER
I20260807 08:34:28.751362 13054 consensus_queue.cc:260] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.751724 13054 ts_tablet_manager.cc:1428] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:34:28.752914 13055 tablet_bootstrap.cc:492] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8: No bootstrap required, opened a new log
I20260807 08:34:28.753026 13055 ts_tablet_manager.cc:1397] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260807 08:34:28.753625 13055 raft_consensus.cc:348] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.753773 13055 raft_consensus.cc:374] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:28.753811 13055 raft_consensus.cc:729] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f574837f91546cdb3002f5d9c3ddcc8, State: Initialized, Role: FOLLOWER
I20260807 08:34:28.753961 13055 consensus_queue.cc:260] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:28.754061 13134 ts_tablet_manager.cc:1905] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:28.754248 13055 ts_tablet_manager.cc:1428] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260807 08:34:28.759475 13134 ts_tablet_manager.cc:1918] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:34:28.759807 13134 log.cc:1196] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/1a26eb88d9c444b89228deff48ac7f70
I20260807 08:34:28.760277 13134 ts_tablet_manager.cc:1939] T 1a26eb88d9c444b89228deff48ac7f70 P 6f574837f91546cdb3002f5d9c3ddcc8: Deleting consensus metadata
I20260807 08:34:28.761991 12327 catalog_manager.cc:4841] TS 6f574837f91546cdb3002f5d9c3ddcc8 (127.9.174.3:37171): tablet 1a26eb88d9c444b89228deff48ac7f70 (table test-table [id=ff36c652951f42a2b89ec66ee5703fc1]) successfully deleted
I20260807 08:34:29.068698 13064 raft_consensus.cc:482] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:34:29.068946 13064 raft_consensus.cc:504] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:29.069622 13064 leader_election.cc:288] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3324942fa45b462488e68cd2fb620407 (127.9.174.1:36079), 4ceece55bcf6455e92ba5d8d74106772 (127.9.174.2:44191)
I20260807 08:34:29.070343 12557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea48a2d65fa342b5bb09f20b33b3243e" candidate_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3324942fa45b462488e68cd2fb620407" is_pre_election: true
I20260807 08:34:29.070343 12761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea48a2d65fa342b5bb09f20b33b3243e" candidate_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ceece55bcf6455e92ba5d8d74106772" is_pre_election: true
I20260807 08:34:29.070571 12761 raft_consensus.cc:2454] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f574837f91546cdb3002f5d9c3ddcc8 in term 0.
I20260807 08:34:29.070572 12557 raft_consensus.cc:2454] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f574837f91546cdb3002f5d9c3ddcc8 in term 0.
I20260807 08:34:29.071056 12865 leader_election.cc:302] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3324942fa45b462488e68cd2fb620407, 6f574837f91546cdb3002f5d9c3ddcc8; no voters:
I20260807 08:34:29.071336 13064 raft_consensus.cc:2781] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:34:29.071421 13064 raft_consensus.cc:482] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:34:29.071485 13064 raft_consensus.cc:3037] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:29.073064 13064 raft_consensus.cc:504] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:29.073591 13064 leader_election.cc:288] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [CANDIDATE]: Term 1 election: Requested vote from peers 3324942fa45b462488e68cd2fb620407 (127.9.174.1:36079), 4ceece55bcf6455e92ba5d8d74106772 (127.9.174.2:44191)
I20260807 08:34:29.074201 12761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea48a2d65fa342b5bb09f20b33b3243e" candidate_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ceece55bcf6455e92ba5d8d74106772"
I20260807 08:34:29.074203 12557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea48a2d65fa342b5bb09f20b33b3243e" candidate_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3324942fa45b462488e68cd2fb620407"
I20260807 08:34:29.074409 12557 raft_consensus.cc:3037] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:29.074409 12761 raft_consensus.cc:3037] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:29.076277 12761 raft_consensus.cc:2454] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f574837f91546cdb3002f5d9c3ddcc8 in term 1.
I20260807 08:34:29.076277 12557 raft_consensus.cc:2454] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f574837f91546cdb3002f5d9c3ddcc8 in term 1.
I20260807 08:34:29.076794 12862 leader_election.cc:302] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ceece55bcf6455e92ba5d8d74106772, 6f574837f91546cdb3002f5d9c3ddcc8; no voters:
I20260807 08:34:29.077090 13064 raft_consensus.cc:2781] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:29.077222 13064 raft_consensus.cc:686] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 LEADER]: Becoming Leader. State: Replica: 6f574837f91546cdb3002f5d9c3ddcc8, State: Running, Role: LEADER
I20260807 08:34:29.077440 13064 consensus_queue.cc:237] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:29.079852 12346 catalog_manager.cc:5495] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f574837f91546cdb3002f5d9c3ddcc8 (127.9.174.3). New cstate: current_term: 1 leader_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } health_report { overall_health: HEALTHY } } }
I20260807 08:34:29.147776 12557 raft_consensus.cc:1261] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407 [term 1 FOLLOWER]: Refusing update from remote peer 6f574837f91546cdb3002f5d9c3ddcc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:29.147776 12761 raft_consensus.cc:1261] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Refusing update from remote peer 6f574837f91546cdb3002f5d9c3ddcc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:29.148273 13064 consensus_queue.cc:1035] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:29.148511 13060 consensus_queue.cc:1035] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 08:34:29.167654 13093 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 08:34:30.747076 12344 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:30.747291 12344 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:30.750648 12344 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08: Sending DeleteTablet for 3 replicas of tablet ea48a2d65fa342b5bb09f20b33b3243e
I20260807 08:34:30.751637 12932 tablet_service.cc:1514] Processing DeleteTablet for tablet ea48a2d65fa342b5bb09f20b33b3243e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:30 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:56592
I20260807 08:34:30.751636 12536 tablet_service.cc:1514] Processing DeleteTablet for tablet ea48a2d65fa342b5bb09f20b33b3243e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:30 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:36072
I20260807 08:34:30.751649 12733 tablet_service.cc:1514] Processing DeleteTablet for tablet ea48a2d65fa342b5bb09f20b33b3243e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:30 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:57980
I20260807 08:34:30.752120 13172 tablet_replica.cc:330] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772: stopping tablet replica
I20260807 08:34:30.752149 13173 tablet_replica.cc:330] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8: stopping tablet replica
I20260807 08:34:30.752331 13173 raft_consensus.cc:2229] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:34:30.752449 13172 raft_consensus.cc:2229] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:34:30.752702 13173 raft_consensus.cc:2258] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:30.752745 13172 raft_consensus.cc:2258] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:30.753736 13172 ts_tablet_manager.cc:1905] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:30.757093 13171 tablet_replica.cc:330] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407: stopping tablet replica
I20260807 08:34:30.757282 13171 raft_consensus.cc:2229] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:34:30.757441 13171 raft_consensus.cc:2258] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:30.757496 13173 ts_tablet_manager.cc:1905] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:30.757543 13172 ts_tablet_manager.cc:1918] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:34:30.757730 13172 log.cc:1196] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/ea48a2d65fa342b5bb09f20b33b3243e
I20260807 08:34:30.758188 13172 ts_tablet_manager.cc:1939] T ea48a2d65fa342b5bb09f20b33b3243e P 4ceece55bcf6455e92ba5d8d74106772: Deleting consensus metadata
I20260807 08:34:30.758350 13171 ts_tablet_manager.cc:1905] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:30.759256 12327 catalog_manager.cc:4841] TS 4ceece55bcf6455e92ba5d8d74106772 (127.9.174.2:44191): tablet ea48a2d65fa342b5bb09f20b33b3243e (table test-table [id=a783ad5cf362442eb95f7a5a65c5a83e]) successfully deleted
I20260807 08:34:30.761482 13173 ts_tablet_manager.cc:1918] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:34:30.761591 13173 log.cc:1196] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/ea48a2d65fa342b5bb09f20b33b3243e
I20260807 08:34:30.761935 13173 ts_tablet_manager.cc:1939] T ea48a2d65fa342b5bb09f20b33b3243e P 6f574837f91546cdb3002f5d9c3ddcc8: Deleting consensus metadata
I20260807 08:34:30.761935 13171 ts_tablet_manager.cc:1918] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:34:30.762012 13171 log.cc:1196] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/wal/wals/ea48a2d65fa342b5bb09f20b33b3243e
I20260807 08:34:30.762487 13171 ts_tablet_manager.cc:1939] T ea48a2d65fa342b5bb09f20b33b3243e P 3324942fa45b462488e68cd2fb620407: Deleting consensus metadata
I20260807 08:34:30.763028 12327 catalog_manager.cc:4841] TS 6f574837f91546cdb3002f5d9c3ddcc8 (127.9.174.3:37171): tablet ea48a2d65fa342b5bb09f20b33b3243e (table test-table [id=a783ad5cf362442eb95f7a5a65c5a83e]) successfully deleted
I20260807 08:34:30.763440 12329 catalog_manager.cc:4841] TS 3324942fa45b462488e68cd2fb620407 (127.9.174.1:36079): tablet ea48a2d65fa342b5bb09f20b33b3243e (table test-table [id=a783ad5cf362442eb95f7a5a65c5a83e]) successfully deleted
Aug 07 08:34:30 dist-test-slave-h3ft krb5kdc[12284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091670, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:34:30 dist-test-slave-h3ft krb5kdc[12284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091670, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:30.789309 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:30.778406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39684 (local address 127.9.174.62:37957)
0807 08:34:30.778866 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:30.778877 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:30.778953 (+ 76us) server_negotiation.cc:407] Connection header received
0807 08:34:30.779235 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:30.779246 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:30.779308 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:30.779418 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:30.780269 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.780938 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.781942 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.782204 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.782307 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:30.785242 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:30.785245 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:30.785248 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:30.785276 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:30.787303 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:30.787744 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:30.787746 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:30.787748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:30.787837 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:30.788168 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:30.788171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:30.788172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:30.788434 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:30.788590 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:30.788884 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 08:34:30.789059 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":226,"threads_started":1}
I20260807 08:34:30.792416 12344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:34:30.792953 12344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:34:30.798975 12733 tablet_service.cc:1467] Processing CreateTablet for tablet 2f04ad075d0143bebc22ad6281b5b20c (DEFAULT_TABLE table=test-table [id=23077d98b5fc404abed3af41dfcf02bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:30.798975 12932 tablet_service.cc:1467] Processing CreateTablet for tablet 2f04ad075d0143bebc22ad6281b5b20c (DEFAULT_TABLE table=test-table [id=23077d98b5fc404abed3af41dfcf02bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:30.799029 12536 tablet_service.cc:1467] Processing CreateTablet for tablet 2f04ad075d0143bebc22ad6281b5b20c (DEFAULT_TABLE table=test-table [id=23077d98b5fc404abed3af41dfcf02bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:30.799397 12932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f04ad075d0143bebc22ad6281b5b20c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:30.799399 12733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f04ad075d0143bebc22ad6281b5b20c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:30.799472 12536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f04ad075d0143bebc22ad6281b5b20c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:30.802716 13186 tablet_bootstrap.cc:492] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772: Bootstrap starting.
I20260807 08:34:30.802883 13185 tablet_bootstrap.cc:492] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8: Bootstrap starting.
I20260807 08:34:30.803239 13187 tablet_bootstrap.cc:492] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407: Bootstrap starting.
I20260807 08:34:30.804008 13185 tablet_bootstrap.cc:654] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:30.804211 13186 tablet_bootstrap.cc:654] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:30.804826 13187 tablet_bootstrap.cc:654] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:30.805405 13185 tablet_bootstrap.cc:492] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8: No bootstrap required, opened a new log
I20260807 08:34:30.805548 13185 ts_tablet_manager.cc:1397] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:34:30.805724 13186 tablet_bootstrap.cc:492] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772: No bootstrap required, opened a new log
I20260807 08:34:30.805843 13186 ts_tablet_manager.cc:1397] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260807 08:34:30.806087 13185 raft_consensus.cc:348] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:30.806257 13185 raft_consensus.cc:374] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:30.806300 13185 raft_consensus.cc:729] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f574837f91546cdb3002f5d9c3ddcc8, State: Initialized, Role: FOLLOWER
I20260807 08:34:30.806403 13186 raft_consensus.cc:348] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:30.806445 13185 consensus_queue.cc:260] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:30.806559 13186 raft_consensus.cc:374] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:30.806599 13186 raft_consensus.cc:729] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ceece55bcf6455e92ba5d8d74106772, State: Initialized, Role: FOLLOWER
I20260807 08:34:30.806646 13187 tablet_bootstrap.cc:492] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407: No bootstrap required, opened a new log
I20260807 08:34:30.806739 13187 ts_tablet_manager.cc:1397] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 08:34:30.806813 13185 ts_tablet_manager.cc:1428] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:34:30.806794 13186 consensus_queue.cc:260] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:30.807315 13186 ts_tablet_manager.cc:1428] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:34:30.807740 13187 raft_consensus.cc:348] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:30.807874 13187 raft_consensus.cc:374] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:30.807910 13187 raft_consensus.cc:729] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3324942fa45b462488e68cd2fb620407, State: Initialized, Role: FOLLOWER
I20260807 08:34:30.808040 13187 consensus_queue.cc:260] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:30.808669 13187 ts_tablet_manager.cc:1428] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 08:34:30.869360 13192 raft_consensus.cc:482] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:34:30.869596 13192 raft_consensus.cc:504] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:30.871297 13192 leader_election.cc:288] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ceece55bcf6455e92ba5d8d74106772 (127.9.174.2:44191), 6f574837f91546cdb3002f5d9c3ddcc8 (127.9.174.3:37171)
I20260807 08:34:30.876982 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:30.871742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:49185 (local address 127.9.174.2:44191)
0807 08:34:30.872213 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:30.872222 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:30.872309 (+ 87us) server_negotiation.cc:407] Connection header received
0807 08:34:30.872561 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:30.872566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:30.872668 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:30.872771 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:30.873268 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.873978 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.875543 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.876280 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.876403 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:30.876482 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:30.876733 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 08:34:30.876825 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":226,"threads_started":1}
I20260807 08:34:30.876997 13195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:30.871622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:44191 (local address 127.9.174.1:49185)
0807 08:34:30.872184 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:30.872205 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:30.872361 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:30.872972 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:30.872976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:30.872993 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:30.873150 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:30.873157 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.874094 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.874099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:30.875387 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:30.875398 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.876476 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.876482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:30.876591 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:30.876617 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 08:34:30.876702 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 08:34:30.876798 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":1,"thread_start_us":256,"threads_started":1}
I20260807 08:34:30.877344 13199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:30.872550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:38987 (local address 127.9.174.3:37171)
0807 08:34:30.872875 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:30.872884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:30.872932 (+ 48us) server_negotiation.cc:407] Connection header received
0807 08:34:30.873101 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:30.873104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:30.873170 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:30.873295 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:30.873573 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.874424 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.875841 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.876421 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.876537 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:30.876628 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:30.877057 (+ 429us) server_negotiation.cc:299] Negotiation successful
0807 08:34:30.877175 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":124,"threads_started":1}
I20260807 08:34:30.877650 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:30.872316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:37171 (local address 127.9.174.1:38987)
0807 08:34:30.872805 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:30.872819 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:30.872972 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:30.873311 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:30.873314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:30.873333 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:30.873456 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:30.873463 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.874560 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.874564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:30.875694 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:30.875702 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.876690 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.876696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:30.876790 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:30.876809 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 08:34:30.876874 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 08:34:30.876952 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":125,"threads_started":1}
I20260807 08:34:30.877714 12761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f04ad075d0143bebc22ad6281b5b20c" candidate_uuid: "3324942fa45b462488e68cd2fb620407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ceece55bcf6455e92ba5d8d74106772" is_pre_election: true
I20260807 08:34:30.877890 12761 raft_consensus.cc:2454] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3324942fa45b462488e68cd2fb620407 in term 0.
I20260807 08:34:30.878103 12961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f04ad075d0143bebc22ad6281b5b20c" candidate_uuid: "3324942fa45b462488e68cd2fb620407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" is_pre_election: true
I20260807 08:34:30.878306 12961 raft_consensus.cc:2454] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3324942fa45b462488e68cd2fb620407 in term 0.
I20260807 08:34:30.878328 12469 leader_election.cc:302] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3324942fa45b462488e68cd2fb620407, 4ceece55bcf6455e92ba5d8d74106772; no voters:
I20260807 08:34:30.878985 13192 raft_consensus.cc:2781] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:34:30.879083 13192 raft_consensus.cc:482] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:34:30.879136 13192 raft_consensus.cc:3037] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:30.880650 13192 raft_consensus.cc:504] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:30.881100 13192 leader_election.cc:288] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [CANDIDATE]: Term 1 election: Requested vote from peers 4ceece55bcf6455e92ba5d8d74106772 (127.9.174.2:44191), 6f574837f91546cdb3002f5d9c3ddcc8 (127.9.174.3:37171)
I20260807 08:34:30.881421 12761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f04ad075d0143bebc22ad6281b5b20c" candidate_uuid: "3324942fa45b462488e68cd2fb620407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ceece55bcf6455e92ba5d8d74106772"
I20260807 08:34:30.881582 12761 raft_consensus.cc:3037] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:30.881560 12961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f04ad075d0143bebc22ad6281b5b20c" candidate_uuid: "3324942fa45b462488e68cd2fb620407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f574837f91546cdb3002f5d9c3ddcc8"
I20260807 08:34:30.881703 12961 raft_consensus.cc:3037] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:30.882827 12961 raft_consensus.cc:2454] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3324942fa45b462488e68cd2fb620407 in term 1.
I20260807 08:34:30.882947 12761 raft_consensus.cc:2454] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3324942fa45b462488e68cd2fb620407 in term 1.
I20260807 08:34:30.883208 12469 leader_election.cc:302] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3324942fa45b462488e68cd2fb620407, 6f574837f91546cdb3002f5d9c3ddcc8; no voters:
I20260807 08:34:30.883406 13192 raft_consensus.cc:2781] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:30.883673 13192 raft_consensus.cc:686] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [term 1 LEADER]: Becoming Leader. State: Replica: 3324942fa45b462488e68cd2fb620407, State: Running, Role: LEADER
I20260807 08:34:30.884006 13192 consensus_queue.cc:237] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:30.886929 12346 catalog_manager.cc:5495] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3324942fa45b462488e68cd2fb620407 (127.9.174.1). New cstate: current_term: 1 leader_uuid: "3324942fa45b462488e68cd2fb620407" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } health_report { overall_health: UNKNOWN } } }
I20260807 08:34:30.917081 13205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:30.911970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55302 (local address 127.9.174.1:36079)
0807 08:34:30.912549 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:30.912563 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:30.912590 (+ 27us) server_negotiation.cc:407] Connection header received
0807 08:34:30.912693 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:30.912695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:30.912781 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:30.912968 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:30.913304 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.914000 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.915148 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.915437 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.915547 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:30.915914 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:30.916520 (+ 606us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:30.916625 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:30.916807 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 08:34:30.916901 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":200,"threads_started":1}
I20260807 08:34:30.930963 13212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:30.926894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:44191 (local address 127.9.174.1:33137)
0807 08:34:30.927344 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:30.927358 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:30.927487 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:30.927766 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:30.927769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:30.927776 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:34:30.927963 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:30.927968 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.928737 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.928740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:30.929486 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:30.929496 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.930012 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.930016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:30.930115 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:30.930176 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:34:30.930576 (+ 400us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:34:30.930582 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:34:30.930639 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 08:34:30.930723 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":201,"threads_started":1}
I20260807 08:34:30.931087 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:30.927108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:33137 (local address 127.9.174.2:44191)
0807 08:34:30.927268 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:30.927272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:30.927451 (+ 179us) server_negotiation.cc:407] Connection header received
0807 08:34:30.927593 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:30.927595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:30.927647 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:30.927738 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:30.928067 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.928620 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.929597 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.929817 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.929898 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:30.930384 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:30.930476 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:30.930560 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:30.930719 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 08:34:30.930818 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:34:30.938372 12761 raft_consensus.cc:1261] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Refusing update from remote peer 3324942fa45b462488e68cd2fb620407: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:30.938488 12961 raft_consensus.cc:1261] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 FOLLOWER]: Refusing update from remote peer 3324942fa45b462488e68cd2fb620407: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:30.938964 13201 consensus_queue.cc:1035] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:30.939278 13192 consensus_queue.cc:1035] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:30.961202 13205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:30.956997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:52887 (local address 127.9.174.1:36079)
0807 08:34:30.957143 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:30.957149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:30.957297 (+ 148us) server_negotiation.cc:407] Connection header received
0807 08:34:30.957453 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:30.957455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:30.957542 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:30.957631 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:30.957987 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.958609 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.959795 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.960123 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.960221 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:30.960620 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:30.960727 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:30.960823 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:30.960938 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 08:34:30.961015 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 08:34:30.961205 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:30.956859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:36079 (local address 127.9.174.2:52887)
0807 08:34:30.957184 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:30.957205 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:30.957346 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:30.957663 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:30.957666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:30.957676 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:34:30.957838 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:30.957847 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.958748 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.958751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:30.959577 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:30.959588 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:30.960274 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:30.960278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:30.960413 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:30.960463 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:34:30.960834 (+ 371us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:34:30.960842 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 08:34:30.960924 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:34:30.961034 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":121,"threads_started":1}
I20260807 08:34:30.986755 12346 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:30.987035 12346 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:30.989689 12346 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08: Sending DeleteTablet for 3 replicas of tablet 2f04ad075d0143bebc22ad6281b5b20c
I20260807 08:34:30.990418 12536 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f04ad075d0143bebc22ad6281b5b20c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:30 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:36072
I20260807 08:34:30.990579 12733 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f04ad075d0143bebc22ad6281b5b20c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:30 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:57980
I20260807 08:34:30.990658 13171 tablet_replica.cc:330] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407: stopping tablet replica
I20260807 08:34:30.990782 13172 tablet_replica.cc:330] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772: stopping tablet replica
I20260807 08:34:30.990796 13171 raft_consensus.cc:2229] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:34:30.990837 12932 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f04ad075d0143bebc22ad6281b5b20c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:30 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:56592
I20260807 08:34:30.990912 13172 raft_consensus.cc:2229] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:34:30.991084 13172 raft_consensus.cc:2258] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:30.991209 13171 raft_consensus.cc:2258] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:30.991308 13173 tablet_replica.cc:330] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8: stopping tablet replica
I20260807 08:34:30.991457 13173 raft_consensus.cc:2229] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:34:30.991585 13173 raft_consensus.cc:2258] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:30.992360 13171 ts_tablet_manager.cc:1905] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:30.992630 13172 ts_tablet_manager.cc:1905] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:30.995398 13172 ts_tablet_manager.cc:1918] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:34:30.995507 13172 log.cc:1196] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/2f04ad075d0143bebc22ad6281b5b20c
I20260807 08:34:30.995810 13172 ts_tablet_manager.cc:1939] T 2f04ad075d0143bebc22ad6281b5b20c P 4ceece55bcf6455e92ba5d8d74106772: Deleting consensus metadata
I20260807 08:34:30.996140 13171 ts_tablet_manager.cc:1918] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:34:30.996227 13171 log.cc:1196] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-0/wal/wals/2f04ad075d0143bebc22ad6281b5b20c
I20260807 08:34:30.996212 13173 ts_tablet_manager.cc:1905] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:30.996572 13171 ts_tablet_manager.cc:1939] T 2f04ad075d0143bebc22ad6281b5b20c P 3324942fa45b462488e68cd2fb620407: Deleting consensus metadata
I20260807 08:34:30.996702 12327 catalog_manager.cc:4841] TS 4ceece55bcf6455e92ba5d8d74106772 (127.9.174.2:44191): tablet 2f04ad075d0143bebc22ad6281b5b20c (table test-table [id=23077d98b5fc404abed3af41dfcf02bf]) successfully deleted
I20260807 08:34:30.997743 12329 catalog_manager.cc:4841] TS 3324942fa45b462488e68cd2fb620407 (127.9.174.1:36079): tablet 2f04ad075d0143bebc22ad6281b5b20c (table test-table [id=23077d98b5fc404abed3af41dfcf02bf]) successfully deleted
I20260807 08:34:30.998272 12346 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:34:30.998828 12346 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:34:30.999414 13173 ts_tablet_manager.cc:1918] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:34:30.999521 13173 log.cc:1196] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/2f04ad075d0143bebc22ad6281b5b20c
I20260807 08:34:30.999814 13173 ts_tablet_manager.cc:1939] T 2f04ad075d0143bebc22ad6281b5b20c P 6f574837f91546cdb3002f5d9c3ddcc8: Deleting consensus metadata
I20260807 08:34:31.000610 12327 catalog_manager.cc:4841] TS 6f574837f91546cdb3002f5d9c3ddcc8 (127.9.174.3:37171): tablet 2f04ad075d0143bebc22ad6281b5b20c (table test-table [id=23077d98b5fc404abed3af41dfcf02bf]) successfully deleted
I20260807 08:34:31.004989 12536 tablet_service.cc:1467] Processing CreateTablet for tablet e3a90f64ab64435ea7c3310382ecc6fc (DEFAULT_TABLE table=test-table [id=bd5b91b5537241518ee40930e1de8791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:31.005380 12536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a90f64ab64435ea7c3310382ecc6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:31.005424 12733 tablet_service.cc:1467] Processing CreateTablet for tablet e3a90f64ab64435ea7c3310382ecc6fc (DEFAULT_TABLE table=test-table [id=bd5b91b5537241518ee40930e1de8791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:31.005889 12733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a90f64ab64435ea7c3310382ecc6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:31.005796 12932 tablet_service.cc:1467] Processing CreateTablet for tablet e3a90f64ab64435ea7c3310382ecc6fc (DEFAULT_TABLE table=test-table [id=bd5b91b5537241518ee40930e1de8791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:31.006285 12932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a90f64ab64435ea7c3310382ecc6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:31.008893 13186 tablet_bootstrap.cc:492] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772: Bootstrap starting.
I20260807 08:34:31.009514 13187 tablet_bootstrap.cc:492] T e3a90f64ab64435ea7c3310382ecc6fc P 3324942fa45b462488e68cd2fb620407: Bootstrap starting.
I20260807 08:34:31.009546 13185 tablet_bootstrap.cc:492] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8: Bootstrap starting.
I20260807 08:34:31.010715 13186 tablet_bootstrap.cc:654] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:31.010825 13185 tablet_bootstrap.cc:654] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:31.011801 13187 tablet_bootstrap.cc:654] T e3a90f64ab64435ea7c3310382ecc6fc P 3324942fa45b462488e68cd2fb620407: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:31.012347 13185 tablet_bootstrap.cc:492] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8: No bootstrap required, opened a new log
I20260807 08:34:31.012466 13185 ts_tablet_manager.cc:1397] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 08:34:31.013085 13185 raft_consensus.cc:348] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:31.013193 13186 tablet_bootstrap.cc:492] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772: No bootstrap required, opened a new log
I20260807 08:34:31.013245 13185 raft_consensus.cc:374] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:31.013285 13186 ts_tablet_manager.cc:1397] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260807 08:34:31.013298 13185 raft_consensus.cc:729] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f574837f91546cdb3002f5d9c3ddcc8, State: Initialized, Role: FOLLOWER
I20260807 08:34:31.013446 13185 consensus_queue.cc:260] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:31.013765 13186 raft_consensus.cc:348] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:31.013911 13186 raft_consensus.cc:374] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:31.013945 13186 raft_consensus.cc:729] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ceece55bcf6455e92ba5d8d74106772, State: Initialized, Role: FOLLOWER
I20260807 08:34:31.013964 13185 ts_tablet_manager.cc:1428] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260807 08:34:31.014074 13186 consensus_queue.cc:260] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:31.014369 13186 ts_tablet_manager.cc:1428] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:34:31.016522 13187 tablet_bootstrap.cc:492] T e3a90f64ab64435ea7c3310382ecc6fc P 3324942fa45b462488e68cd2fb620407: No bootstrap required, opened a new log
I20260807 08:34:31.016626 13187 ts_tablet_manager.cc:1397] T e3a90f64ab64435ea7c3310382ecc6fc P 3324942fa45b462488e68cd2fb620407: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260807 08:34:31.017686 13187 raft_consensus.cc:348] T e3a90f64ab64435ea7c3310382ecc6fc P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:31.017788 13187 raft_consensus.cc:374] T e3a90f64ab64435ea7c3310382ecc6fc P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:31.017810 13187 raft_consensus.cc:729] T e3a90f64ab64435ea7c3310382ecc6fc P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3324942fa45b462488e68cd2fb620407, State: Initialized, Role: FOLLOWER
I20260807 08:34:31.017917 13187 consensus_queue.cc:260] T e3a90f64ab64435ea7c3310382ecc6fc P 3324942fa45b462488e68cd2fb620407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:31.018268 13187 ts_tablet_manager.cc:1428] T e3a90f64ab64435ea7c3310382ecc6fc P 3324942fa45b462488e68cd2fb620407: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 08:34:31.203804 13144 raft_consensus.cc:482] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:34:31.204092 13144 raft_consensus.cc:504] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:31.206090 13144 leader_election.cc:288] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3324942fa45b462488e68cd2fb620407 (127.9.174.1:36079), 6f574837f91546cdb3002f5d9c3ddcc8 (127.9.174.3:37171)
I20260807 08:34:31.206812 12556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a90f64ab64435ea7c3310382ecc6fc" candidate_uuid: "4ceece55bcf6455e92ba5d8d74106772" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3324942fa45b462488e68cd2fb620407" is_pre_election: true
I20260807 08:34:31.206938 12961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a90f64ab64435ea7c3310382ecc6fc" candidate_uuid: "4ceece55bcf6455e92ba5d8d74106772" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" is_pre_election: true
I20260807 08:34:31.207059 12556 raft_consensus.cc:2454] T e3a90f64ab64435ea7c3310382ecc6fc P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ceece55bcf6455e92ba5d8d74106772 in term 0.
I20260807 08:34:31.207139 12961 raft_consensus.cc:2454] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ceece55bcf6455e92ba5d8d74106772 in term 0.
I20260807 08:34:31.207544 12667 leader_election.cc:302] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3324942fa45b462488e68cd2fb620407, 4ceece55bcf6455e92ba5d8d74106772, 6f574837f91546cdb3002f5d9c3ddcc8; no voters:
I20260807 08:34:31.207777 13144 raft_consensus.cc:2781] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:34:31.207846 13144 raft_consensus.cc:482] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:34:31.207929 13144 raft_consensus.cc:3037] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:31.209394 13144 raft_consensus.cc:504] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:31.209816 13144 leader_election.cc:288] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [CANDIDATE]: Term 1 election: Requested vote from peers 3324942fa45b462488e68cd2fb620407 (127.9.174.1:36079), 6f574837f91546cdb3002f5d9c3ddcc8 (127.9.174.3:37171)
I20260807 08:34:31.210218 12556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a90f64ab64435ea7c3310382ecc6fc" candidate_uuid: "4ceece55bcf6455e92ba5d8d74106772" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3324942fa45b462488e68cd2fb620407"
I20260807 08:34:31.210218 12961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a90f64ab64435ea7c3310382ecc6fc" candidate_uuid: "4ceece55bcf6455e92ba5d8d74106772" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f574837f91546cdb3002f5d9c3ddcc8"
I20260807 08:34:31.210390 12556 raft_consensus.cc:3037] T e3a90f64ab64435ea7c3310382ecc6fc P 3324942fa45b462488e68cd2fb620407 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:31.210403 12961 raft_consensus.cc:3037] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:31.212141 12556 raft_consensus.cc:2454] T e3a90f64ab64435ea7c3310382ecc6fc P 3324942fa45b462488e68cd2fb620407 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ceece55bcf6455e92ba5d8d74106772 in term 1.
I20260807 08:34:31.212133 12961 raft_consensus.cc:2454] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ceece55bcf6455e92ba5d8d74106772 in term 1.
I20260807 08:34:31.212524 12672 leader_election.cc:302] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3324942fa45b462488e68cd2fb620407, 4ceece55bcf6455e92ba5d8d74106772; no voters:
I20260807 08:34:31.212790 13144 raft_consensus.cc:2781] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:31.212932 13144 raft_consensus.cc:686] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [term 1 LEADER]: Becoming Leader. State: Replica: 4ceece55bcf6455e92ba5d8d74106772, State: Running, Role: LEADER
I20260807 08:34:31.213119 13144 consensus_queue.cc:237] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } }
I20260807 08:34:31.216105 12344 catalog_manager.cc:5495] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ceece55bcf6455e92ba5d8d74106772 (127.9.174.2). New cstate: current_term: 1 leader_uuid: "4ceece55bcf6455e92ba5d8d74106772" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ceece55bcf6455e92ba5d8d74106772" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 } health_report { overall_health: UNKNOWN } } }
I20260807 08:34:31.236533 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:31.231659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55046 (local address 127.9.174.2:44191)
0807 08:34:31.232095 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:31.232102 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:31.232119 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:34:31.232176 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:31.232180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:31.232237 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:31.232382 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:31.232772 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:31.233826 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:31.235238 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:31.235495 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:31.235608 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:31.235942 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:31.236044 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:31.236123 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:31.236254 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 08:34:31.236317 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337}
I20260807 08:34:31.249217 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:31.245536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:44191 (local address 127.9.174.2:44847)
0807 08:34:31.245771 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:31.245787 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:31.245899 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:31.246150 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:31.246153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:31.246159 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:34:31.246264 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:31.246269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:31.247208 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:31.247212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:31.247884 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:31.247893 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:31.248427 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:31.248430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:31.248558 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:31.248593 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:34:31.248885 (+ 292us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:34:31.248890 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:34:31.248946 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 08:34:31.249045 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":19}
I20260807 08:34:31.249217 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:31.245789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:44847 (local address 127.9.174.2:44191)
0807 08:34:31.245941 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:31.245945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:31.245959 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:34:31.246003 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:31.246005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:31.246053 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:31.246126 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:31.246435 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:31.247095 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:31.248022 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:31.248295 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:31.248411 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:31.248704 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:31.248802 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:31.248868 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:31.248976 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 08:34:31.249037 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":68}
I20260807 08:34:31.257264 12961 raft_consensus.cc:1261] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 FOLLOWER]: Refusing update from remote peer 4ceece55bcf6455e92ba5d8d74106772: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:31.257364 12556 raft_consensus.cc:1261] T e3a90f64ab64435ea7c3310382ecc6fc P 3324942fa45b462488e68cd2fb620407 [term 1 FOLLOWER]: Refusing update from remote peer 4ceece55bcf6455e92ba5d8d74106772: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:31.257776 13154 consensus_queue.cc:1035] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3324942fa45b462488e68cd2fb620407" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:31.258042 13203 consensus_queue.cc:1035] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f574837f91546cdb3002f5d9c3ddcc8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 08:34:31.276984 13146 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 08:34:32.845893 12346 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:32.846151 12346 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:32.850116 12346 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45ae3ee37adf4f30851ad671b00a0f08: Sending DeleteTablet for 3 replicas of tablet e3a90f64ab64435ea7c3310382ecc6fc
I20260807 08:34:32.851140 12536 tablet_service.cc:1514] Processing DeleteTablet for tablet e3a90f64ab64435ea7c3310382ecc6fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:32 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:36072
I20260807 08:34:32.851426 12932 tablet_service.cc:1514] Processing DeleteTablet for tablet e3a90f64ab64435ea7c3310382ecc6fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:32 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:56592
I20260807 08:34:32.851671 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 12444
I20260807 08:34:32.854939 12733 tablet_service.cc:1514] Processing DeleteTablet for tablet e3a90f64ab64435ea7c3310382ecc6fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:32 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:57980
I20260807 08:34:32.857975 13265 tablet_replica.cc:330] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8: stopping tablet replica
I20260807 08:34:32.858222 13265 raft_consensus.cc:2229] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:34:32.858402 13265 raft_consensus.cc:2258] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:32.860865 13265 ts_tablet_manager.cc:1905] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:32.861325 13266 tablet_replica.cc:330] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772: stopping tablet replica
I20260807 08:34:32.861558 13266 raft_consensus.cc:2229] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:34:32.862082 13266 raft_consensus.cc:2258] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:32.862890 13266 ts_tablet_manager.cc:1905] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:32.864696 13265 ts_tablet_manager.cc:1918] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:34:32.864792 13265 log.cc:1196] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/e3a90f64ab64435ea7c3310382ecc6fc
I20260807 08:34:32.865087 13265 ts_tablet_manager.cc:1939] T e3a90f64ab64435ea7c3310382ecc6fc P 6f574837f91546cdb3002f5d9c3ddcc8: Deleting consensus metadata
I20260807 08:34:32.866236 13266 ts_tablet_manager.cc:1918] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:34:32.866328 13266 log.cc:1196] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/e3a90f64ab64435ea7c3310382ecc6fc
I20260807 08:34:32.866312 12327 catalog_manager.cc:4841] TS 6f574837f91546cdb3002f5d9c3ddcc8 (127.9.174.3:37171): tablet e3a90f64ab64435ea7c3310382ecc6fc (table test-table [id=bd5b91b5537241518ee40930e1de8791]) successfully deleted
I20260807 08:34:32.866662 13266 ts_tablet_manager.cc:1939] T e3a90f64ab64435ea7c3310382ecc6fc P 4ceece55bcf6455e92ba5d8d74106772: Deleting consensus metadata
I20260807 08:34:32.867575 12327 catalog_manager.cc:4841] TS 4ceece55bcf6455e92ba5d8d74106772 (127.9.174.2:44191): tablet e3a90f64ab64435ea7c3310382ecc6fc (table test-table [id=bd5b91b5537241518ee40930e1de8791]) successfully deleted
W20260807 08:34:32.875633 12329 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:36079 (error 108)
I20260807 08:34:32.876467 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 12642
I20260807 08:34:32.886548 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:32.885915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:36079 (local address 127.0.0.1:55304)
0807 08:34:32.886312 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:32.886418 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:36079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":152,"threads_started":1}
W20260807 08:34:32.886790 12329 catalog_manager.cc:4584] TS 3324942fa45b462488e68cd2fb620407 (127.9.174.1:36079): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e3a90f64ab64435ea7c3310382ecc6fc: Network error: Client connection negotiation failed: client connection to 127.9.174.1:36079: connect: Connection refused (error 111)
I20260807 08:34:32.896152 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 12837
I20260807 08:34:32.911988 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 12302
2026-08-07T08:34:32Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7417 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:34:33 dist-test-slave-h3ft krb5kdc[13272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:34:33 dist-test-slave-h3ft krb5kdc[13272](info): set up 2 sockets
Aug 07 08:34:33 dist-test-slave-h3ft krb5kdc[13272](info): commencing operation
krb5kdc: starting...
W20260807 08:34:34.231868 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:34:34 dist-test-slave-h3ft krb5kdc[13272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:34:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:34:34.386560 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:43249
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:43249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:34.524472 13295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:34.524899 13295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:34.524988 13295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:34.529418 13295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:34.529548 13295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:34.529584 13295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:34.529623 13295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:34.529685 13295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:34.535784 13295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:43249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:43249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:34.537168 13295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:34.538470 13295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:34.549223 13307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:34.549256 13304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:34.549433 13295 server_base.cc:1034] running on GCE node
W20260807 08:34:34.549214 13303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:34.550283 13295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:34.551851 13295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:34.553061 13295 hybrid_clock.cc:648] HybridClock initialized: now 1786091674553024 us; error 52 us; skew 500 ppm
Aug 07 08:34:34 dist-test-slave-h3ft krb5kdc[13272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091674, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:34.556907 13295 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:34:34.558109 13295 webserver.cc:460] Webserver started at http://127.9.174.62:33211/ using document root <none> and password file <none>
I20260807 08:34:34.558830 13295 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:34.558919 13295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:34.559239 13295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:34.562319 13295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "c8b508e38c82472ca6095ff90ba9f8ca"
format_stamp: "Formatted at 2026-08-07 08:34:34 on dist-test-slave-h3ft"
server_key: "585e63a384eb9282e3943814077e1911"
server_key_iv: "9cfb1199584de4ffeaa862ed7c77dc95"
server_key_version: "encryptionkey@0"
I20260807 08:34:34.563050 13295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "c8b508e38c82472ca6095ff90ba9f8ca"
format_stamp: "Formatted at 2026-08-07 08:34:34 on dist-test-slave-h3ft"
server_key: "585e63a384eb9282e3943814077e1911"
server_key_iv: "9cfb1199584de4ffeaa862ed7c77dc95"
server_key_version: "encryptionkey@0"
I20260807 08:34:34.567688 13295 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:34:34.570761 13316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:34.572526 13295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:34:34.572701 13295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "c8b508e38c82472ca6095ff90ba9f8ca"
format_stamp: "Formatted at 2026-08-07 08:34:34 on dist-test-slave-h3ft"
server_key: "585e63a384eb9282e3943814077e1911"
server_key_iv: "9cfb1199584de4ffeaa862ed7c77dc95"
server_key_version: "encryptionkey@0"
I20260807 08:34:34.572829 13295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:34.618003 13295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:34.622215 13295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:34.622449 13295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:34.631836 13295 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:43249
I20260807 08:34:34.631902 13391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:43249 every 8 connection(s)
I20260807 08:34:34.633189 13295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:34.636842 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 13295
I20260807 08:34:34.637037 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:34:34.637240 13392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:34.637418 9912 external_mini_cluster.cc:1428] Setting key 72744989aec1b8a8c9be123e2d54333b
I20260807 08:34:34.646523 13392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca: Bootstrap starting.
Aug 07 08:34:34 dist-test-slave-h3ft krb5kdc[13272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:34.649936 13392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:34.650890 13392 log.cc:824] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:34.653528 13392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca: No bootstrap required, opened a new log
I20260807 08:34:34.654428 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:34.639294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36182 (local address 127.9.174.62:43249)
0807 08:34:34.639895 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:34.639914 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:34.639962 (+ 48us) server_negotiation.cc:407] Connection header received
0807 08:34:34.640778 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:34.640800 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:34.641228 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:34.641639 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:34.642710 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:34.643905 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:34.645151 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:34.645727 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:34.645856 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:34.649032 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:34.649039 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:34.649056 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:34.649092 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:34.651010 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:34.651601 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:34.651612 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:34.651617 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:34.651749 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:34.652059 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:34.652064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:34.652066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:34.652567 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:34.652798 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:34.653115 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 08:34:34.653471 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":179,"threads_started":1}
I20260807 08:34:34.658321 13392 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b508e38c82472ca6095ff90ba9f8ca" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43249 } }
I20260807 08:34:34.658608 13392 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:34.658699 13392 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8b508e38c82472ca6095ff90ba9f8ca, State: Initialized, Role: FOLLOWER
I20260807 08:34:34.659255 13392 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b508e38c82472ca6095ff90ba9f8ca" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43249 } }
I20260807 08:34:34.659428 13392 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:34:34.659499 13392 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:34:34.659610 13392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:34.660974 13392 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b508e38c82472ca6095ff90ba9f8ca" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43249 } }
I20260807 08:34:34.661396 13392 leader_election.cc:302] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8b508e38c82472ca6095ff90ba9f8ca; no voters:
I20260807 08:34:34.661839 13392 leader_election.cc:288] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:34:34.661974 13398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:34.662240 13398 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [term 1 LEADER]: Becoming Leader. State: Replica: c8b508e38c82472ca6095ff90ba9f8ca, State: Running, Role: LEADER
I20260807 08:34:34.662657 13398 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b508e38c82472ca6095ff90ba9f8ca" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43249 } }
I20260807 08:34:34.663559 13392 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:34.664757 13400 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8b508e38c82472ca6095ff90ba9f8ca. Latest consensus state: current_term: 1 leader_uuid: "c8b508e38c82472ca6095ff90ba9f8ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b508e38c82472ca6095ff90ba9f8ca" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43249 } } }
I20260807 08:34:34.664929 13400 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:34.665292 13399 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8b508e38c82472ca6095ff90ba9f8ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b508e38c82472ca6095ff90ba9f8ca" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43249 } } }
I20260807 08:34:34.665406 13399 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:34.665542 13411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:34.669648 13411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:34.677553 13411 catalog_manager.cc:1347] Generated new cluster ID: fe2c3c55194c44fab39f52b29160c295
I20260807 08:34:34.677654 13411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:34.686484 13411 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:34.688045 13411 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:34.705353 13411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8b508e38c82472ca6095ff90ba9f8ca: Generated new TSK 0
I20260807 08:34:34.706552 13411 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:34:34.744807 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43249
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:34.883703 13427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:34.884075 13427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:34.884160 13427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:34.888379 13427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:34.888476 13427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:34.888573 13427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:34:34.894358 13427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:34.895947 13427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:34.897298 13427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:34.909401 13434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:34.909448 13433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:34.909401 13438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:34.909741 13427 server_base.cc:1034] running on GCE node
I20260807 08:34:34.910337 13427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:34.911316 13427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:34.912508 13427 hybrid_clock.cc:648] HybridClock initialized: now 1786091674912466 us; error 56 us; skew 500 ppm
Aug 07 08:34:34 dist-test-slave-h3ft krb5kdc[13272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091674, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:34.915966 13427 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:34:34.917109 13427 webserver.cc:460] Webserver started at http://127.9.174.1:45167/ using document root <none> and password file <none>
I20260807 08:34:34.917793 13427 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:34.917860 13427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:34.918071 13427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:34.920686 13427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "60f41b913060449f9e1697caf1bbdc14"
format_stamp: "Formatted at 2026-08-07 08:34:34 on dist-test-slave-h3ft"
server_key: "b20617dbded0c80e13ca2e5bf21f58e8"
server_key_iv: "8c27b6447ca1320036cf67f3a2d97bde"
server_key_version: "encryptionkey@0"
I20260807 08:34:34.921314 13427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "60f41b913060449f9e1697caf1bbdc14"
format_stamp: "Formatted at 2026-08-07 08:34:34 on dist-test-slave-h3ft"
server_key: "b20617dbded0c80e13ca2e5bf21f58e8"
server_key_iv: "8c27b6447ca1320036cf67f3a2d97bde"
server_key_version: "encryptionkey@0"
I20260807 08:34:34.926111 13427 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:34:34.929410 13446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:34.931252 13427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:34:34.931421 13427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "60f41b913060449f9e1697caf1bbdc14"
format_stamp: "Formatted at 2026-08-07 08:34:34 on dist-test-slave-h3ft"
server_key: "b20617dbded0c80e13ca2e5bf21f58e8"
server_key_iv: "8c27b6447ca1320036cf67f3a2d97bde"
server_key_version: "encryptionkey@0"
I20260807 08:34:34.931576 13427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:34.955812 13427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:34.959640 13427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:34.959903 13427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:34.960737 13427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:34.961838 13427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:34.961894 13427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:34.961967 13427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:34.962000 13427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:34.973420 13427 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:34015
I20260807 08:34:34.973440 13604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:34015 every 8 connection(s)
I20260807 08:34:34.974570 13427 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:34:34.974987 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 13427
I20260807 08:34:34.975183 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:34:34.975587 9912 external_mini_cluster.cc:1428] Setting key 982c3df1f4fae22439e00471d83572c2
Aug 07 08:34:34 dist-test-slave-h3ft krb5kdc[13272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091674, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
I20260807 08:34:34.992460 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:34.976816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:53533 (local address 127.9.174.62:43249)
0807 08:34:34.977000 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:34.977006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:34.978689 (+ 1683us) server_negotiation.cc:407] Connection header received
0807 08:34:34.980422 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:34.980426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:34.980520 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:34.980640 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:34.982243 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:34.982904 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:34.984433 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:34.984724 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:34.984851 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:34.988068 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:34.988072 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:34.988074 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:34.988106 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:34.989827 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:34.990435 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:34.990439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:34.990441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:34.990539 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:34.990928 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:34.990940 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:34.990942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:34.991234 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:34.991351 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:34.992157 (+ 806us) server_negotiation.cc:299] Negotiation successful
0807 08:34:34.992293 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 08:34:34.994115 13607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:34.977437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43249 (local address 127.9.174.1:53533)
0807 08:34:34.978486 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:34.978563 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:34.980111 (+ 1548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:34.980920 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:34.980930 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:34.981405 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:34.982058 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:34.982075 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:34.983091 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:34.983095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:34.984286 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:34.984298 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:34.984915 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:34.984920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:34.985140 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:34.985945 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:34.985966 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:34.987856 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:34.989978 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:34.989986 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:34.990006 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:34.990311 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:34.990640 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:34.990645 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:34.990647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:34.990814 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:34.991352 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:34.991362 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:34.991809 (+ 447us) client_negotiation.cc:769] Sending connection context
0807 08:34:34.992071 (+ 262us) client_negotiation.cc:240] Negotiation successful
0807 08:34:34.992596 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":151,"threads_started":1}
I20260807 08:34:34.995788 13605 heartbeater.cc:347] Connected to a master server at 127.9.174.62:43249
I20260807 08:34:34.996134 13605 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:34.996837 13605 heartbeater.cc:511] Master 127.9.174.62:43249 requested a full tablet report, sending...
I20260807 08:34:34.998836 13342 ts_manager.cc:194] Registered new tserver with Master: 60f41b913060449f9e1697caf1bbdc14 (127.9.174.1:34015)
I20260807 08:34:35.000345 13342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:53533
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:34:35.010085 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43249
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:35.146725 13611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:35.147162 13611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:35.147256 13611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:35.151547 13611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:35.151609 13611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:35.151672 13611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:34:35.157696 13611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:35.159233 13611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:35.160528 13611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:35.172960 13620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:35.172936 13618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:35.173000 13617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:35.173543 13611 server_base.cc:1034] running on GCE node
I20260807 08:34:35.174113 13611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:35.175076 13611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:35.176302 13611 hybrid_clock.cc:648] HybridClock initialized: now 1786091675176271 us; error 43 us; skew 500 ppm
Aug 07 08:34:35 dist-test-slave-h3ft krb5kdc[13272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091675, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:35.180164 13611 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:34:35.181373 13611 webserver.cc:460] Webserver started at http://127.9.174.2:35335/ using document root <none> and password file <none>
I20260807 08:34:35.182104 13611 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:35.182210 13611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:35.182514 13611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:35.185237 13611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "83917b2bb7d4437daefe3e6d43167a81"
format_stamp: "Formatted at 2026-08-07 08:34:35 on dist-test-slave-h3ft"
server_key: "863b46dbda8ac18519c7ebea63829037"
server_key_iv: "72f1b1c92467bc80905088ae0c4b643d"
server_key_version: "encryptionkey@0"
I20260807 08:34:35.185979 13611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "83917b2bb7d4437daefe3e6d43167a81"
format_stamp: "Formatted at 2026-08-07 08:34:35 on dist-test-slave-h3ft"
server_key: "863b46dbda8ac18519c7ebea63829037"
server_key_iv: "72f1b1c92467bc80905088ae0c4b643d"
server_key_version: "encryptionkey@0"
I20260807 08:34:35.191215 13611 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 08:34:35.194664 13627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:35.196429 13611 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:34:35.196611 13611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "83917b2bb7d4437daefe3e6d43167a81"
format_stamp: "Formatted at 2026-08-07 08:34:35 on dist-test-slave-h3ft"
server_key: "863b46dbda8ac18519c7ebea63829037"
server_key_iv: "72f1b1c92467bc80905088ae0c4b643d"
server_key_version: "encryptionkey@0"
I20260807 08:34:35.196766 13611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:35.219412 13611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:35.223657 13611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:35.223932 13611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:35.224763 13611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:35.226006 13611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:35.226091 13611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:35.226173 13611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:35.226212 13611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:35.238303 13611 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:39651
I20260807 08:34:35.238325 13788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:39651 every 8 connection(s)
I20260807 08:34:35.239552 13611 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:34:35.239828 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 13611
I20260807 08:34:35.239960 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:34:35.240319 9912 external_mini_cluster.cc:1428] Setting key ac116cf1f0a0ebaf33edc1c049a8ba1d
Aug 07 08:34:35 dist-test-slave-h3ft krb5kdc[13272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091675, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
I20260807 08:34:35.257974 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:35.242009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:59541 (local address 127.9.174.62:43249)
0807 08:34:35.242177 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:35.242181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:35.243777 (+ 1596us) server_negotiation.cc:407] Connection header received
0807 08:34:35.245408 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:35.245413 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:35.245495 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:35.245553 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:35.247312 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:35.247900 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:35.249881 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:35.250124 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:35.250211 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:35.253713 (+ 3502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:35.253719 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:35.253723 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:35.253765 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:35.255557 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:35.256168 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:35.256171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:35.256172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:35.256265 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:35.256651 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:35.256655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:35.256657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:35.256883 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:35.256997 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:35.257595 (+ 598us) server_negotiation.cc:299] Negotiation successful
0807 08:34:35.257753 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:34:35.259373 13791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:35.242519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43249 (local address 127.9.174.2:59541)
0807 08:34:35.243554 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:35.243629 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:35.245070 (+ 1441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:35.245975 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:35.245983 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:35.246478 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:35.247156 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:35.247173 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:35.248073 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:35.248079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:35.249733 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:35.249750 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:35.250324 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:35.250332 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:35.250616 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:35.251460 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:35.251491 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:35.253499 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:35.255682 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:35.255688 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:35.255706 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:35.256033 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:35.256361 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:35.256364 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:35.256366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:35.256545 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:35.257002 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:35.257012 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:35.257322 (+ 310us) client_negotiation.cc:769] Sending connection context
0807 08:34:35.257533 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 08:34:35.257935 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"thread_start_us":149,"threads_started":1}
I20260807 08:34:35.260859 13789 heartbeater.cc:347] Connected to a master server at 127.9.174.62:43249
I20260807 08:34:35.261196 13789 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:35.261960 13789 heartbeater.cc:511] Master 127.9.174.62:43249 requested a full tablet report, sending...
I20260807 08:34:35.263409 13342 ts_manager.cc:194] Registered new tserver with Master: 83917b2bb7d4437daefe3e6d43167a81 (127.9.174.2:39651)
I20260807 08:34:35.264156 13342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:59541
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:34:35.278753 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43249
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:35.416899 13798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:35.417274 13798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:35.417327 13798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:35.421718 13798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:35.421808 13798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:35.421916 13798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:34:35.428062 13798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:35.429637 13798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:35.431059 13798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:35.444211 13807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:35.444936 13812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:35.445606 13798 server_base.cc:1034] running on GCE node
W20260807 08:34:35.450614 13808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:35.451581 13798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:35.452617 13798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:35.453833 13798 hybrid_clock.cc:648] HybridClock initialized: now 1786091675453806 us; error 42 us; skew 500 ppm
Aug 07 08:34:35 dist-test-slave-h3ft krb5kdc[13272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091675, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:35.457729 13798 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:34:35.459123 13798 webserver.cc:460] Webserver started at http://127.9.174.3:42179/ using document root <none> and password file <none>
I20260807 08:34:35.459899 13798 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:35.460004 13798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:35.460309 13798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:35.463196 13798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "d2fc0339311e44cca7e5d4bc4f81155c"
format_stamp: "Formatted at 2026-08-07 08:34:35 on dist-test-slave-h3ft"
server_key: "4d25128ad360ed4e3e3609ad8004dc11"
server_key_iv: "8aca2a5cddbffd86589024edbfee45a9"
server_key_version: "encryptionkey@0"
I20260807 08:34:35.463981 13798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "d2fc0339311e44cca7e5d4bc4f81155c"
format_stamp: "Formatted at 2026-08-07 08:34:35 on dist-test-slave-h3ft"
server_key: "4d25128ad360ed4e3e3609ad8004dc11"
server_key_iv: "8aca2a5cddbffd86589024edbfee45a9"
server_key_version: "encryptionkey@0"
I20260807 08:34:35.469173 13798 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:34:35.472783 13820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:35.474434 13798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 08:34:35.474632 13798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "d2fc0339311e44cca7e5d4bc4f81155c"
format_stamp: "Formatted at 2026-08-07 08:34:35 on dist-test-slave-h3ft"
server_key: "4d25128ad360ed4e3e3609ad8004dc11"
server_key_iv: "8aca2a5cddbffd86589024edbfee45a9"
server_key_version: "encryptionkey@0"
I20260807 08:34:35.474797 13798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:35.497298 13798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:35.501945 13798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:35.502233 13798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:35.503057 13798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:35.504168 13798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:35.504235 13798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:35.504287 13798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:35.504309 13798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:35.515496 13798 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:37013
I20260807 08:34:35.515523 13989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:37013 every 8 connection(s)
I20260807 08:34:35.516693 13798 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:34:35.519402 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 13798
I20260807 08:34:35.519552 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:34:35.519933 9912 external_mini_cluster.cc:1428] Setting key 670f38a0f94ac764141c2387aa2ef63b
Aug 07 08:34:35 dist-test-slave-h3ft krb5kdc[13272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091675, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:35.533564 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:35.519325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:51739 (local address 127.9.174.62:43249)
0807 08:34:35.519541 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:35.519546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:35.520947 (+ 1401us) server_negotiation.cc:407] Connection header received
0807 08:34:35.522311 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:35.522315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:35.522417 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:35.522518 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:35.524108 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:35.524717 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:35.526005 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:35.526272 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:35.526370 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:35.529483 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:35.529486 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:35.529488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:35.529528 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:35.531098 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:35.531715 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:35.531717 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:35.531719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:35.531811 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:35.532208 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:35.532210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:35.532211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:35.532425 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:35.532510 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:35.533189 (+ 679us) server_negotiation.cc:299] Negotiation successful
0807 08:34:35.533346 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 08:34:35.535004 13992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:35.519778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43249 (local address 127.9.174.3:51739)
0807 08:34:35.520751 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:35.520826 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:35.522040 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:35.522759 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:35.522768 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:35.523315 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:35.523947 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:35.523963 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:35.524848 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:35.524851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:35.525871 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:35.525881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:35.526461 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:35.526464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:35.526644 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:35.527470 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:35.527491 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:35.529270 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:35.531229 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:35.531236 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:35.531261 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:35.531611 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:35.531929 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:35.531932 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:35.531934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:35.532112 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:35.532533 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:35.532539 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:35.532867 (+ 328us) client_negotiation.cc:769] Sending connection context
0807 08:34:35.533099 (+ 232us) client_negotiation.cc:240] Negotiation successful
0807 08:34:35.533605 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":189,"threads_started":1}
I20260807 08:34:35.536321 13990 heartbeater.cc:347] Connected to a master server at 127.9.174.62:43249
I20260807 08:34:35.536652 13990 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:35.537443 13990 heartbeater.cc:511] Master 127.9.174.62:43249 requested a full tablet report, sending...
I20260807 08:34:35.538720 13342 ts_manager.cc:194] Registered new tserver with Master: d2fc0339311e44cca7e5d4bc4f81155c (127.9.174.3:37013)
I20260807 08:34:35.539433 13342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:51739
I20260807 08:34:35.547104 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 08:34:35.565320 13396 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:35.563311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36196 (local address 127.9.174.62:43249)
0807 08:34:35.563512 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:35.563520 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:35.563636 (+ 116us) server_negotiation.cc:407] Connection header received
0807 08:34:35.563897 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:35.563901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:35.563972 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:35.564075 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:35.565114 (+ 1039us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36196: BlockingRecv error: recv got EOF from 127.0.0.1:36196 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 08:34:35.569623 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 13427
I20260807 08:34:35.599282 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 13611
I20260807 08:34:35.616797 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 13798
I20260807 08:34:35.635780 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 13295
2026-08-07T08:34:35Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2724 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:34:35 dist-test-slave-h3ft krb5kdc[14017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:34:35 dist-test-slave-h3ft krb5kdc[14017](info): set up 2 sockets
Aug 07 08:34:35 dist-test-slave-h3ft krb5kdc[14017](info): commencing operation
krb5kdc: starting...
W20260807 08:34:36.921097 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:34:37 dist-test-slave-h3ft krb5kdc[14017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:34:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
W20260807 08:34:37.939221 9912 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.9.174.62: real 0.889s user 0.000s sys 0.005s
I20260807 08:34:37.939960 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:42577
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:44435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:42577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:38.083635 14036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:38.084021 14036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:38.084069 14036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:38.088384 14036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:38.088521 14036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:38.088557 14036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:38.088588 14036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:38.088614 14036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:38.094847 14036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:44435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:42577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:42577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:38.096367 14036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:38.097645 14036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:38.108784 14041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:38.108798 14043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:38.108804 14047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:38.109171 14036 server_base.cc:1034] running on GCE node
I20260807 08:34:38.110049 14036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:38.111645 14036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:38.112886 14036 hybrid_clock.cc:648] HybridClock initialized: now 1786091678112829 us; error 69 us; skew 500 ppm
Aug 07 08:34:38 dist-test-slave-h3ft krb5kdc[14017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091678, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:38.121675 14036 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:34:38.122987 14036 webserver.cc:460] Webserver started at http://127.9.174.62:42659/ using document root <none> and password file <none>
I20260807 08:34:38.123847 14036 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:38.123957 14036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:38.124253 14036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:38.127202 14036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "8a73f6629f5547689ce249a6ad80ee24"
format_stamp: "Formatted at 2026-08-07 08:34:38 on dist-test-slave-h3ft"
server_key: "f1cb5dbab13afdfd0f23fd1c777f34ac"
server_key_iv: "df649671fb6f248b656749a7c682dea2"
server_key_version: "encryptionkey@0"
I20260807 08:34:38.127940 14036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "8a73f6629f5547689ce249a6ad80ee24"
format_stamp: "Formatted at 2026-08-07 08:34:38 on dist-test-slave-h3ft"
server_key: "f1cb5dbab13afdfd0f23fd1c777f34ac"
server_key_iv: "df649671fb6f248b656749a7c682dea2"
server_key_version: "encryptionkey@0"
I20260807 08:34:38.133162 14036 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:34:38.136588 14056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:38.138384 14036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:34:38.138584 14036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "8a73f6629f5547689ce249a6ad80ee24"
format_stamp: "Formatted at 2026-08-07 08:34:38 on dist-test-slave-h3ft"
server_key: "f1cb5dbab13afdfd0f23fd1c777f34ac"
server_key_iv: "df649671fb6f248b656749a7c682dea2"
server_key_version: "encryptionkey@0"
I20260807 08:34:38.138754 14036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:38.159195 14036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:38.163502 14036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:38.163792 14036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:38.174014 14036 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:42577
I20260807 08:34:38.174379 14133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:42577 every 8 connection(s)
I20260807 08:34:38.175596 14036 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:38.179476 14136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:38.181012 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 14036
I20260807 08:34:38.181175 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:34:38.181519 9912 external_mini_cluster.cc:1428] Setting key dbe177909b10d7d72509d7365d551e86
I20260807 08:34:38.188120 14136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24: Bootstrap starting.
I20260807 08:34:38.191565 14136 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:38.192507 14136 log.cc:824] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:34:38 dist-test-slave-h3ft krb5kdc[14017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:38.195255 14136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24: No bootstrap required, opened a new log
I20260807 08:34:38.199476 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:38.183351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50274 (local address 127.9.174.62:42577)
0807 08:34:38.183920 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:38.183945 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:38.184000 (+ 55us) server_negotiation.cc:407] Connection header received
0807 08:34:38.184968 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:38.184998 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:38.185524 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:38.185990 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:38.187412 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:38.188974 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:38.189950 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:38.190685 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:38.190852 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:38.194139 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:38.194145 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:38.194161 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:38.194190 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:38.196097 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:38.196593 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:38.196598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:38.196604 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:38.196722 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:38.197028 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:38.197038 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:38.197039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:38.197504 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:38.197729 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:38.198065 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 08:34:38.198404 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":144,"threads_started":1}
I20260807 08:34:38.199626 14136 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a73f6629f5547689ce249a6ad80ee24" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42577 } }
I20260807 08:34:38.199978 14136 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:38.200050 14136 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a73f6629f5547689ce249a6ad80ee24, State: Initialized, Role: FOLLOWER
I20260807 08:34:38.200618 14136 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a73f6629f5547689ce249a6ad80ee24" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42577 } }
I20260807 08:34:38.200842 14136 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:34:38.200933 14136 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:34:38.201046 14136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:38.203161 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:34:38.224786 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:38.218693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50280 (local address 127.9.174.62:42577)
0807 08:34:38.218892 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:38.218899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:38.219068 (+ 169us) server_negotiation.cc:407] Connection header received
0807 08:34:38.219275 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:38.219279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:38.219362 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:38.219478 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:38.220579 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:38.221189 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:38.222121 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:38.222398 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:38.222525 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:38.223510 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:38.223514 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:38.223517 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:34:38.224207 (+ 690us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:38.224244 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:34:38.224263 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:38.224374 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:38.224510 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 08:34:38.224593 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 08:34:38.234804 14136 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a73f6629f5547689ce249a6ad80ee24" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42577 } }
I20260807 08:34:38.235625 14136 leader_election.cc:302] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a73f6629f5547689ce249a6ad80ee24; no voters:
I20260807 08:34:38.236166 14136 leader_election.cc:288] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:34:38.236349 14151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:38.236680 14151 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [term 1 LEADER]: Becoming Leader. State: Replica: 8a73f6629f5547689ce249a6ad80ee24, State: Running, Role: LEADER
I20260807 08:34:38.237243 14151 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a73f6629f5547689ce249a6ad80ee24" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42577 } }
I20260807 08:34:38.238168 14136 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:38.239559 14152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a73f6629f5547689ce249a6ad80ee24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a73f6629f5547689ce249a6ad80ee24" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42577 } } }
I20260807 08:34:38.239746 14152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:38.240110 14155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a73f6629f5547689ce249a6ad80ee24. Latest consensus state: current_term: 1 leader_uuid: "8a73f6629f5547689ce249a6ad80ee24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a73f6629f5547689ce249a6ad80ee24" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42577 } } }
I20260807 08:34:38.240254 14155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:38.240399 14159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:38.245709 14159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:38.253274 14159 catalog_manager.cc:1347] Generated new cluster ID: eeff61e6d97b4893b2c916a455c0fafb
I20260807 08:34:38.253391 14159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:38.273731 14159 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:38.274951 14159 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:38.290025 14159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a73f6629f5547689ce249a6ad80ee24: Generated new TSK 0
I20260807 08:34:38.291208 14159 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:34:38.299957 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 14036
2026-08-07T08:34:38Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2668 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:34:38 dist-test-slave-h3ft krb5kdc[14180](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:34:38 dist-test-slave-h3ft krb5kdc[14180](info): set up 2 sockets
Aug 07 08:34:38 dist-test-slave-h3ft krb5kdc[14180](info): commencing operation
krb5kdc: starting...
W20260807 08:34:40.061805 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.722s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:34:40 dist-test-slave-h3ft krb5kdc[14180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:34:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:34:40.215513 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:42331
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:39999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:42331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:40.357921 14202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:40.358376 14202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:40.358464 14202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:40.362938 14202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:40.363081 14202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:40.363124 14202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:40.363185 14202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:40.363241 14202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:40.369357 14202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:39999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:42331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:42331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:40.370921 14202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:40.372292 14202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:40.383318 14214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:40.383455 14202 server_base.cc:1034] running on GCE node
W20260807 08:34:40.383318 14216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:40.383318 14213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:40.384455 14202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:40.385946 14202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:40.387151 14202 hybrid_clock.cc:648] HybridClock initialized: now 1786091680387110 us; error 58 us; skew 500 ppm
Aug 07 08:34:40 dist-test-slave-h3ft krb5kdc[14180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091680, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:40.390610 14202 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:34:40.391819 14202 webserver.cc:460] Webserver started at http://127.9.174.62:38103/ using document root <none> and password file <none>
I20260807 08:34:40.392544 14202 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:40.392661 14202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:40.392954 14202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:40.395673 14202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "cf64464188d24b51a76ba9537b0d4038"
format_stamp: "Formatted at 2026-08-07 08:34:40 on dist-test-slave-h3ft"
server_key: "d905395a6b6fff1150a76fec14057407"
server_key_iv: "fcabbfc172ea3dfb632f328dd3ddcca9"
server_key_version: "encryptionkey@0"
I20260807 08:34:40.396452 14202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "cf64464188d24b51a76ba9537b0d4038"
format_stamp: "Formatted at 2026-08-07 08:34:40 on dist-test-slave-h3ft"
server_key: "d905395a6b6fff1150a76fec14057407"
server_key_iv: "fcabbfc172ea3dfb632f328dd3ddcca9"
server_key_version: "encryptionkey@0"
I20260807 08:34:40.401618 14202 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:34:40.405062 14228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:40.406816 14202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:34:40.407073 14202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "cf64464188d24b51a76ba9537b0d4038"
format_stamp: "Formatted at 2026-08-07 08:34:40 on dist-test-slave-h3ft"
server_key: "d905395a6b6fff1150a76fec14057407"
server_key_iv: "fcabbfc172ea3dfb632f328dd3ddcca9"
server_key_version: "encryptionkey@0"
I20260807 08:34:40.407234 14202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:40.428665 14202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:40.433142 14202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:40.433559 14202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:40.442880 14202 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:42331
I20260807 08:34:40.443287 14297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:42331 every 8 connection(s)
I20260807 08:34:40.444217 14202 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:40.446023 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 14202
I20260807 08:34:40.446183 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:34:40.446571 9912 external_mini_cluster.cc:1428] Setting key f32f13704145d53b7a8d45c63e2f5e2d
I20260807 08:34:40.448705 14298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:40.457199 14298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038: Bootstrap starting.
Aug 07 08:34:40 dist-test-slave-h3ft krb5kdc[14180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:40.460239 14298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:40.461414 14298 log.cc:824] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:40.464354 14298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038: No bootstrap required, opened a new log
I20260807 08:34:40.464449 14303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:40.448359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58178 (local address 127.9.174.62:42331)
0807 08:34:40.448859 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:40.448879 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:40.448928 (+ 49us) server_negotiation.cc:407] Connection header received
0807 08:34:40.449717 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:40.449737 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:40.450128 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:40.450471 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:40.451554 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:40.452764 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:40.454047 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:40.454655 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:40.454816 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:40.458271 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:40.458278 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:40.458294 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:40.458329 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:40.460330 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:40.460977 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:40.460984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:40.460989 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:40.461141 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:40.461500 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:40.461504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:40.461506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:40.462353 (+ 847us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:40.462623 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:40.463074 (+ 451us) server_negotiation.cc:299] Negotiation successful
0807 08:34:40.463484 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":165,"threads_started":1}
I20260807 08:34:40.468952 14298 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf64464188d24b51a76ba9537b0d4038" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42331 } }
I20260807 08:34:40.469259 14298 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:40.469352 14298 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf64464188d24b51a76ba9537b0d4038, State: Initialized, Role: FOLLOWER
I20260807 08:34:40.469937 14298 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf64464188d24b51a76ba9537b0d4038" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42331 } }
I20260807 08:34:40.470129 14298 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:34:40.470216 14298 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:34:40.470326 14298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:40.471619 14298 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf64464188d24b51a76ba9537b0d4038" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42331 } }
I20260807 08:34:40.472038 14298 leader_election.cc:302] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf64464188d24b51a76ba9537b0d4038; no voters:
I20260807 08:34:40.472517 14298 leader_election.cc:288] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:34:40.472708 14310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:40.472973 14310 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [term 1 LEADER]: Becoming Leader. State: Replica: cf64464188d24b51a76ba9537b0d4038, State: Running, Role: LEADER
I20260807 08:34:40.473368 14310 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf64464188d24b51a76ba9537b0d4038" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42331 } }
I20260807 08:34:40.474431 14298 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:40.475028 14310 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf64464188d24b51a76ba9537b0d4038. Latest consensus state: current_term: 1 leader_uuid: "cf64464188d24b51a76ba9537b0d4038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf64464188d24b51a76ba9537b0d4038" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42331 } } }
I20260807 08:34:40.475169 14310 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:40.475647 14316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:40.476389 14312 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf64464188d24b51a76ba9537b0d4038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf64464188d24b51a76ba9537b0d4038" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42331 } } }
I20260807 08:34:40.476684 14312 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:40.479715 14316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:40.487062 14316 catalog_manager.cc:1347] Generated new cluster ID: 470b620999734b5f9d54c54178663c34
I20260807 08:34:40.487165 14316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:40.506500 14316 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:40.508199 14316 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:40.521181 14316 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf64464188d24b51a76ba9537b0d4038: Generated new TSK 0
I20260807 08:34:40.522362 14316 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:34:40.559736 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42331
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:39999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:40.702720 14336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:40.703192 14336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:40.703286 14336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:40.707792 14336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:40.707887 14336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:40.708006 14336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:34:40.714398 14336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:39999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:40.715863 14336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:40.717211 14336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:40.729022 14347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:40.729048 14351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:40.729035 14348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:40.729239 14336 server_base.cc:1034] running on GCE node
I20260807 08:34:40.729804 14336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:40.730592 14336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:40.731784 14336 hybrid_clock.cc:648] HybridClock initialized: now 1786091680731764 us; error 35 us; skew 500 ppm
Aug 07 08:34:40 dist-test-slave-h3ft krb5kdc[14180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091680, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:40.735296 14336 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:34:40.736438 14336 webserver.cc:460] Webserver started at http://127.9.174.1:42037/ using document root <none> and password file <none>
I20260807 08:34:40.737143 14336 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:40.737213 14336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:40.737475 14336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:40.740146 14336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "211c5427141d4d6180451d4da9924284"
format_stamp: "Formatted at 2026-08-07 08:34:40 on dist-test-slave-h3ft"
server_key: "c3c11ff1285e90f13d2bdf1747a27e2d"
server_key_iv: "8c0e966cb8eda7b9985e5006d0f8abb8"
server_key_version: "encryptionkey@0"
I20260807 08:34:40.740801 14336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "211c5427141d4d6180451d4da9924284"
format_stamp: "Formatted at 2026-08-07 08:34:40 on dist-test-slave-h3ft"
server_key: "c3c11ff1285e90f13d2bdf1747a27e2d"
server_key_iv: "8c0e966cb8eda7b9985e5006d0f8abb8"
server_key_version: "encryptionkey@0"
I20260807 08:34:40.745397 14336 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 08:34:40.748494 14360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:40.750101 14336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 08:34:40.750258 14336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "211c5427141d4d6180451d4da9924284"
format_stamp: "Formatted at 2026-08-07 08:34:40 on dist-test-slave-h3ft"
server_key: "c3c11ff1285e90f13d2bdf1747a27e2d"
server_key_iv: "8c0e966cb8eda7b9985e5006d0f8abb8"
server_key_version: "encryptionkey@0"
I20260807 08:34:40.750394 14336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:40.762679 14336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:40.766103 14336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:40.766332 14336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:40.767128 14336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:40.768477 14336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:40.768532 14336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:40.768579 14336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:40.768599 14336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:40.780198 14336 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:36577
I20260807 08:34:40.780220 14517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:36577 every 8 connection(s)
I20260807 08:34:40.781319 14336 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:34:40.790261 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 14336
I20260807 08:34:40.790397 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:34:40.790786 9912 external_mini_cluster.cc:1428] Setting key e9eb35db0274badb1701f53d6d885407
Aug 07 08:34:40 dist-test-slave-h3ft krb5kdc[14180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091680, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:40.799116 14303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:40.783582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:58931 (local address 127.9.174.62:42331)
0807 08:34:40.783759 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:40.783763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:40.784763 (+ 1000us) server_negotiation.cc:407] Connection header received
0807 08:34:40.786358 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:40.786362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:40.786429 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:40.786510 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:40.788184 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:40.788792 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:40.790230 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:40.790623 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:40.790741 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:40.794241 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:40.794244 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:40.794247 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:40.794277 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:40.796086 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:40.796919 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:40.796922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:40.796924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:40.797029 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:40.797534 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:40.797537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:40.797539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:40.797773 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:40.797865 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:40.798696 (+ 831us) server_negotiation.cc:299] Negotiation successful
0807 08:34:40.798850 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 08:34:40.800633 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:40.784054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42331 (local address 127.9.174.1:58931)
0807 08:34:40.784586 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:40.784637 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:40.786124 (+ 1487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:40.786705 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:40.786712 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:40.787273 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:40.787991 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:40.788010 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:40.788911 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:40.788913 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:40.790057 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:40.790072 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:40.790824 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:40.790830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:40.791081 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:40.791895 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:40.791934 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:40.794012 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:40.796242 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:40.796253 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:40.796279 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:40.796745 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:40.797154 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:40.797162 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:40.797166 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:40.797413 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:40.797899 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:40.797909 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:40.798368 (+ 459us) client_negotiation.cc:769] Sending connection context
0807 08:34:40.798640 (+ 272us) client_negotiation.cc:240] Negotiation successful
0807 08:34:40.799140 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":134,"threads_started":1}
I20260807 08:34:40.802450 14518 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42331
I20260807 08:34:40.802785 14518 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:40.803514 14518 heartbeater.cc:511] Master 127.9.174.62:42331 requested a full tablet report, sending...
I20260807 08:34:40.805545 14248 ts_manager.cc:194] Registered new tserver with Master: 211c5427141d4d6180451d4da9924284 (127.9.174.1:36577)
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
I20260807 08:34:40.807104 14248 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:58931
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:34:40.831584 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42331
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:39999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:40.985193 14526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:40.985606 14526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:40.985695 14526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:40.990402 14526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:40.990499 14526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:40.990592 14526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:34:40.997346 14526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:39999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:40.998925 14526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:41.000454 14526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:41.012423 14531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:41.012423 14537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:41.012480 14532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:41.012826 14526 server_base.cc:1034] running on GCE node
I20260807 08:34:41.013350 14526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:41.014154 14526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:41.015398 14526 hybrid_clock.cc:648] HybridClock initialized: now 1786091681015363 us; error 36 us; skew 500 ppm
Aug 07 08:34:41 dist-test-slave-h3ft krb5kdc[14180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091681, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:41.018760 14526 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:34:41.019913 14526 webserver.cc:460] Webserver started at http://127.9.174.2:36581/ using document root <none> and password file <none>
I20260807 08:34:41.020671 14526 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:41.020772 14526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:41.021060 14526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:41.023744 14526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "45337e1f18d94c5b8410d971e17f9346"
format_stamp: "Formatted at 2026-08-07 08:34:41 on dist-test-slave-h3ft"
server_key: "140b0be43a9c0a73ddae17c65edd3452"
server_key_iv: "f53f292f4cd4fa66869814d5b308f2c7"
server_key_version: "encryptionkey@0"
I20260807 08:34:41.024451 14526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "45337e1f18d94c5b8410d971e17f9346"
format_stamp: "Formatted at 2026-08-07 08:34:41 on dist-test-slave-h3ft"
server_key: "140b0be43a9c0a73ddae17c65edd3452"
server_key_iv: "f53f292f4cd4fa66869814d5b308f2c7"
server_key_version: "encryptionkey@0"
I20260807 08:34:41.029310 14526 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 08:34:41.032785 14546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:41.034590 14526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:34:41.034781 14526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "45337e1f18d94c5b8410d971e17f9346"
format_stamp: "Formatted at 2026-08-07 08:34:41 on dist-test-slave-h3ft"
server_key: "140b0be43a9c0a73ddae17c65edd3452"
server_key_iv: "f53f292f4cd4fa66869814d5b308f2c7"
server_key_version: "encryptionkey@0"
I20260807 08:34:41.034936 14526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:41.055485 14526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:41.059589 14526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:41.059842 14526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:41.060631 14526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:41.061705 14526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:41.061785 14526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:41.061885 14526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:41.061940 14526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:41.073163 14526 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:46401
I20260807 08:34:41.073187 14708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:46401 every 8 connection(s)
I20260807 08:34:41.074385 14526 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:34:41.082212 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 14526
I20260807 08:34:41.082355 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:34:41.082744 9912 external_mini_cluster.cc:1428] Setting key 3e2121ce10b62059f7843dec74f71e78
Aug 07 08:34:41 dist-test-slave-h3ft krb5kdc[14180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091681, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:41.093369 14303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:41.076993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:41199 (local address 127.9.174.62:42331)
0807 08:34:41.077183 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:41.077187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:41.078307 (+ 1120us) server_negotiation.cc:407] Connection header received
0807 08:34:41.079961 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:41.079965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:41.080028 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:41.080114 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:41.081605 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:41.082573 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:41.084285 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:41.084629 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:41.084771 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:41.088605 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:41.088608 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:41.088611 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:41.088641 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:41.090321 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:41.091143 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:41.091146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:41.091148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:41.091247 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:41.091789 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:41.091792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:41.091793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:41.092067 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:41.092160 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:41.093060 (+ 900us) server_negotiation.cc:299] Negotiation successful
0807 08:34:41.093199 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 08:34:41.095103 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:41.077496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42331 (local address 127.9.174.2:41199)
0807 08:34:41.078111 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:41.078164 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:41.079689 (+ 1525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:41.080310 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:41.080317 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:41.080770 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:41.081429 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:41.081445 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:41.082752 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:41.082755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:41.084127 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:41.084140 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:41.084837 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:41.084841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:41.085081 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:41.085910 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:41.085939 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:41.088384 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:41.090467 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:41.090478 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:41.090502 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:41.090961 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:41.091373 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:41.091380 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:41.091385 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:41.091634 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:41.092203 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:41.092212 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:41.092695 (+ 483us) client_negotiation.cc:769] Sending connection context
0807 08:34:41.092985 (+ 290us) client_negotiation.cc:240] Negotiation successful
0807 08:34:41.093509 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":148,"threads_started":1}
I20260807 08:34:41.096792 14709 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42331
I20260807 08:34:41.097127 14709 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
I20260807 08:34:41.097818 14709 heartbeater.cc:511] Master 127.9.174.62:42331 requested a full tablet report, sending...
I20260807 08:34:41.099262 14248 ts_manager.cc:194] Registered new tserver with Master: 45337e1f18d94c5b8410d971e17f9346 (127.9.174.2:46401)
I20260807 08:34:41.100069 14248 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:41199
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:34:41.119453 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42331
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:39999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:41.256938 14718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:41.257308 14718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:41.257392 14718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:41.261742 14718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:41.261852 14718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:41.261960 14718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:34:41.267971 14718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:39999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:41.269425 14718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:41.270745 14718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:41.283139 14723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:41.283181 14724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:41.283074 14726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:41.283319 14718 server_base.cc:1034] running on GCE node
I20260807 08:34:41.283874 14718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:41.284718 14718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:41.285889 14718 hybrid_clock.cc:648] HybridClock initialized: now 1786091681285852 us; error 50 us; skew 500 ppm
Aug 07 08:34:41 dist-test-slave-h3ft krb5kdc[14180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091681, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:41.289392 14718 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:34:41.290529 14718 webserver.cc:460] Webserver started at http://127.9.174.3:39489/ using document root <none> and password file <none>
I20260807 08:34:41.291273 14718 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:41.291347 14718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:41.291581 14718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:41.294118 14718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "27e8080f6db4494db43f964dad261881"
format_stamp: "Formatted at 2026-08-07 08:34:41 on dist-test-slave-h3ft"
server_key: "2c7cd944755de9ba0255841bc6670375"
server_key_iv: "a40bf8b054385b6bc8b258918be7306d"
server_key_version: "encryptionkey@0"
I20260807 08:34:41.294760 14718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "27e8080f6db4494db43f964dad261881"
format_stamp: "Formatted at 2026-08-07 08:34:41 on dist-test-slave-h3ft"
server_key: "2c7cd944755de9ba0255841bc6670375"
server_key_iv: "a40bf8b054385b6bc8b258918be7306d"
server_key_version: "encryptionkey@0"
I20260807 08:34:41.299522 14718 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:34:41.302788 14737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:41.304658 14718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:34:41.304800 14718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "27e8080f6db4494db43f964dad261881"
format_stamp: "Formatted at 2026-08-07 08:34:41 on dist-test-slave-h3ft"
server_key: "2c7cd944755de9ba0255841bc6670375"
server_key_iv: "a40bf8b054385b6bc8b258918be7306d"
server_key_version: "encryptionkey@0"
I20260807 08:34:41.304939 14718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:41.331146 14718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:41.335511 14718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:41.335724 14718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:41.336478 14718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:41.337604 14718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:41.337661 14718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:41.337710 14718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:41.337726 14718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:41.349304 14718 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:35373
I20260807 08:34:41.349339 14893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:35373 every 8 connection(s)
I20260807 08:34:41.350512 14718 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:34:41.360081 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 14718
I20260807 08:34:41.360232 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:34:41.360684 9912 external_mini_cluster.cc:1428] Setting key 0656f36e5f77c390287fae31ec4d295f
Aug 07 08:34:41 dist-test-slave-h3ft krb5kdc[14180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091681, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:41.368913 14303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:41.353113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:41151 (local address 127.9.174.62:42331)
0807 08:34:41.353349 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:41.353353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:41.354517 (+ 1164us) server_negotiation.cc:407] Connection header received
0807 08:34:41.356489 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:41.356494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:41.356579 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:41.356696 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:41.358300 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:41.358898 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:41.360363 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:41.360618 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:41.360686 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:41.364380 (+ 3694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:41.364384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:41.364386 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:41.364419 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:41.366210 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:41.366870 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:41.366882 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:41.366884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:41.366983 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:41.367464 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:41.367468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:41.367470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:41.367709 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:41.367834 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:41.368500 (+ 666us) server_negotiation.cc:299] Negotiation successful
0807 08:34:41.368664 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260807 08:34:41.370461 14896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:41.353566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42331 (local address 127.9.174.3:41151)
0807 08:34:41.354308 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:41.354385 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:41.356200 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:41.356948 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:41.356958 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:41.357450 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:41.358107 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:41.358125 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:41.359084 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:41.359088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:41.360182 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:41.360199 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:41.360971 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:41.360973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:41.361217 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:41.362031 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:41.362052 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:41.364154 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:41.366361 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:41.366369 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:41.366391 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:41.366727 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:41.367145 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:41.367149 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:41.367152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:41.367344 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:41.367845 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:41.367856 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:41.368204 (+ 348us) client_negotiation.cc:769] Sending connection context
0807 08:34:41.368437 (+ 233us) client_negotiation.cc:240] Negotiation successful
0807 08:34:41.368919 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":139,"threads_started":1}
I20260807 08:34:41.372162 14894 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42331
I20260807 08:34:41.372473 14894 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:41.373107 14894 heartbeater.cc:511] Master 127.9.174.62:42331 requested a full tablet report, sending...
I20260807 08:34:41.374540 14248 ts_manager.cc:194] Registered new tserver with Master: 27e8080f6db4494db43f964dad261881 (127.9.174.3:35373)
I20260807 08:34:41.375509 14248 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:41151
I20260807 08:34:41.376047 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:34:41 dist-test-slave-h3ft krb5kdc[14180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091681, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 08:34:41 dist-test-slave-h3ft krb5kdc[14180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091681, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:41.410499 14303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:41.400099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58186 (local address 127.9.174.62:42331)
0807 08:34:41.400325 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:41.400330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:41.400404 (+ 74us) server_negotiation.cc:407] Connection header received
0807 08:34:41.400622 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:41.400626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:41.400690 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:41.400779 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:41.401670 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:41.402340 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:41.403330 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:41.403568 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:41.403660 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:41.406557 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:41.406561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:41.406564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:41.406596 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:41.408319 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:41.408891 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:41.408895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:41.408896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:41.408989 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:41.409268 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:41.409271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:41.409272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:41.409604 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:41.409736 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:41.410092 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 08:34:41.410251 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
W20260807 08:34:41.411099 14248 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58186
I20260807 08:34:41.412447 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 14336
I20260807 08:34:41.433452 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 14526
I20260807 08:34:41.452838 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 14718
I20260807 08:34:41.467738 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 14202
2026-08-07T08:34:41Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3163 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:34:41 dist-test-slave-h3ft krb5kdc[14910](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:34:41 dist-test-slave-h3ft krb5kdc[14910](info): set up 2 sockets
Aug 07 08:34:41 dist-test-slave-h3ft krb5kdc[14910](info): commencing operation
krb5kdc: starting...
W20260807 08:34:42.778440 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:34:42 dist-test-slave-h3ft krb5kdc[14910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:34:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:34:42.943379 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:37305
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:42989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:37305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:43.080600 14931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:43.080997 14931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:43.081089 14931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:43.085433 14931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:43.085548 14931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:43.085587 14931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:43.085651 14931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:43.085691 14931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:43.091531 14931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:42989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:37305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:37305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:43.092911 14931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:43.094219 14931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:43.104920 14941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:43.105095 14931 server_base.cc:1034] running on GCE node
W20260807 08:34:43.104920 14939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:43.105291 14938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:43.106247 14931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:43.108028 14931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:43.109333 14931 hybrid_clock.cc:648] HybridClock initialized: now 1786091683109282 us; error 59 us; skew 500 ppm
Aug 07 08:34:43 dist-test-slave-h3ft krb5kdc[14910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091683, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:43.114032 14931 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:34:43.115624 14931 webserver.cc:460] Webserver started at http://127.9.174.62:38291/ using document root <none> and password file <none>
I20260807 08:34:43.116858 14931 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:43.117197 14931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:43.117544 14931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:43.120728 14931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "4981fbb904194b949a69463c71768651"
format_stamp: "Formatted at 2026-08-07 08:34:43 on dist-test-slave-h3ft"
server_key: "103811ae42390a84f01dad2cd9cff781"
server_key_iv: "5693a8a69efea340dacada0ec1bc0cb7"
server_key_version: "encryptionkey@0"
I20260807 08:34:43.121762 14931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "4981fbb904194b949a69463c71768651"
format_stamp: "Formatted at 2026-08-07 08:34:43 on dist-test-slave-h3ft"
server_key: "103811ae42390a84f01dad2cd9cff781"
server_key_iv: "5693a8a69efea340dacada0ec1bc0cb7"
server_key_version: "encryptionkey@0"
I20260807 08:34:43.127727 14931 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 08:34:43.132153 14950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:43.134527 14931 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 08:34:43.134817 14931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "4981fbb904194b949a69463c71768651"
format_stamp: "Formatted at 2026-08-07 08:34:43 on dist-test-slave-h3ft"
server_key: "103811ae42390a84f01dad2cd9cff781"
server_key_iv: "5693a8a69efea340dacada0ec1bc0cb7"
server_key_version: "encryptionkey@0"
I20260807 08:34:43.135102 14931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:43.168102 14931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:43.172456 14931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:43.172753 14931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:43.181758 14931 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:37305
I20260807 08:34:43.181931 15027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:37305 every 8 connection(s)
I20260807 08:34:43.183041 14931 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:43.184608 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 14931
I20260807 08:34:43.184760 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:34:43.185086 9912 external_mini_cluster.cc:1428] Setting key 3a123b84681320aeda378706f3e5ddab
I20260807 08:34:43.187534 15028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 08:34:43 dist-test-slave-h3ft krb5kdc[14910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:43.196900 15028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651: Bootstrap starting.
I20260807 08:34:43.200027 15028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:43.201130 15028 log.cc:824] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:43.203120 15033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:43.186824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49712 (local address 127.9.174.62:37305)
0807 08:34:43.187513 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:43.187531 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:43.187580 (+ 49us) server_negotiation.cc:407] Connection header received
0807 08:34:43.188452 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:43.188472 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:43.188888 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:43.189247 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:43.190313 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:43.192099 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:43.193381 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:43.193924 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:43.194120 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:43.197519 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:43.197524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:43.197540 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:43.197574 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:43.199581 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:43.200191 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:43.200202 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:43.200206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:43.200330 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:43.200658 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:43.200660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:43.200661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:43.201138 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:43.201353 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:43.201670 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 08:34:43.201983 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":213,"threads_started":1}
I20260807 08:34:43.204193 15028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651: No bootstrap required, opened a new log
I20260807 08:34:43.208433 15028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4981fbb904194b949a69463c71768651" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37305 } }
I20260807 08:34:43.208746 15028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:43.208806 15028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4981fbb904194b949a69463c71768651, State: Initialized, Role: FOLLOWER
I20260807 08:34:43.209371 15028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4981fbb904194b949a69463c71768651" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37305 } }
I20260807 08:34:43.209547 15028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:34:43.209618 15028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:34:43.209745 15028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:43.210875 15028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4981fbb904194b949a69463c71768651" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37305 } }
I20260807 08:34:43.211292 15028 leader_election.cc:302] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4981fbb904194b949a69463c71768651; no voters:
I20260807 08:34:43.211658 15028 leader_election.cc:288] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:34:43.211959 15038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:43.212705 15038 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [term 1 LEADER]: Becoming Leader. State: Replica: 4981fbb904194b949a69463c71768651, State: Running, Role: LEADER
I20260807 08:34:43.213140 15038 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4981fbb904194b949a69463c71768651" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37305 } }
I20260807 08:34:43.213389 15028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:43.215648 15040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4981fbb904194b949a69463c71768651" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4981fbb904194b949a69463c71768651" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37305 } } }
I20260807 08:34:43.215823 15040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:43.216053 15041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4981fbb904194b949a69463c71768651. Latest consensus state: current_term: 1 leader_uuid: "4981fbb904194b949a69463c71768651" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4981fbb904194b949a69463c71768651" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37305 } } }
I20260807 08:34:43.216692 15041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:43.216300 15063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:43.220516 15063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:43.227784 15063 catalog_manager.cc:1347] Generated new cluster ID: 29ce153a752a454a9f6163c219b99ebd
I20260807 08:34:43.227882 15063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:43.256160 15063 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:43.257493 15063 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:43.265744 15063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4981fbb904194b949a69463c71768651: Generated new TSK 0
I20260807 08:34:43.266717 15063 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:34:43.315361 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37305
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:42989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:43.463711 15070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:43.464102 15070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:43.464184 15070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:43.468678 15070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:43.468786 15070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:43.468894 15070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:34:43.475093 15070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:42989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:43.476655 15070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:43.478052 15070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:43.491150 15077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:43.491218 15081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:43.491155 15076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:43.491747 15070 server_base.cc:1034] running on GCE node
I20260807 08:34:43.492317 15070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:43.493117 15070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:43.494283 15070 hybrid_clock.cc:648] HybridClock initialized: now 1786091683494269 us; error 25 us; skew 500 ppm
Aug 07 08:34:43 dist-test-slave-h3ft krb5kdc[14910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091683, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:43.497812 15070 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:34:43.498978 15070 webserver.cc:460] Webserver started at http://127.9.174.1:34937/ using document root <none> and password file <none>
I20260807 08:34:43.499763 15070 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:43.499836 15070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:43.500054 15070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:43.502733 15070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "f70be5334b9244f986e7b7dae6ac097e"
format_stamp: "Formatted at 2026-08-07 08:34:43 on dist-test-slave-h3ft"
server_key: "6cbcdd5da2d194862266bead28d2004f"
server_key_iv: "0e128d5602132a68a88aaab6d5cc5ff6"
server_key_version: "encryptionkey@0"
I20260807 08:34:43.503476 15070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "f70be5334b9244f986e7b7dae6ac097e"
format_stamp: "Formatted at 2026-08-07 08:34:43 on dist-test-slave-h3ft"
server_key: "6cbcdd5da2d194862266bead28d2004f"
server_key_iv: "0e128d5602132a68a88aaab6d5cc5ff6"
server_key_version: "encryptionkey@0"
I20260807 08:34:43.508158 15070 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 08:34:43.511615 15092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:43.513242 15070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:34:43.513370 15070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "f70be5334b9244f986e7b7dae6ac097e"
format_stamp: "Formatted at 2026-08-07 08:34:43 on dist-test-slave-h3ft"
server_key: "6cbcdd5da2d194862266bead28d2004f"
server_key_iv: "0e128d5602132a68a88aaab6d5cc5ff6"
server_key_version: "encryptionkey@0"
I20260807 08:34:43.513496 15070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:43.542979 15070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:43.547174 15070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:43.547385 15070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:43.548136 15070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:43.549170 15070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:43.549223 15070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:43.549273 15070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:43.549288 15070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:43.560171 15070 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:34219
I20260807 08:34:43.560204 15251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:34219 every 8 connection(s)
I20260807 08:34:43.561311 15070 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:34:43.562006 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 15070
I20260807 08:34:43.562160 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:34:43.562610 9912 external_mini_cluster.cc:1428] Setting key 4696f77788fbbeac084c948702f82a65
Aug 07 08:34:43 dist-test-slave-h3ft krb5kdc[14910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091683, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
I20260807 08:34:43.579509 15033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:43.563793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:58895 (local address 127.9.174.62:37305)
0807 08:34:43.563993 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:43.563998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:43.565351 (+ 1353us) server_negotiation.cc:407] Connection header received
0807 08:34:43.566944 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:43.566947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:43.567061 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:43.567175 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:43.568613 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:43.569211 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:43.570488 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:43.570789 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:43.570911 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:43.574905 (+ 3994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:43.574908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:43.574911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:43.574940 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:43.576687 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:43.577316 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:43.577320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:43.577322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:43.577467 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:43.577885 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:43.577887 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:43.577888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:43.578124 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:43.578212 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:43.579133 (+ 921us) server_negotiation.cc:299] Negotiation successful
0807 08:34:43.579282 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
I20260807 08:34:43.580847 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:43.564515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37305 (local address 127.9.174.1:58895)
0807 08:34:43.565182 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:43.565235 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:43.566714 (+ 1479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:43.567397 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:43.567406 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:43.567846 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:43.568441 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:43.568457 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:43.569351 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:43.569360 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:43.570349 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:43.570359 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:43.571054 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:43.571064 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:43.571366 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:43.572240 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:43.572276 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:43.574670 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:43.576831 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:43.576840 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:43.576862 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:43.577198 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:43.577602 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:43.577605 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:43.577607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:43.577786 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:43.578263 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:43.578271 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:43.578716 (+ 445us) client_negotiation.cc:769] Sending connection context
0807 08:34:43.579043 (+ 327us) client_negotiation.cc:240] Negotiation successful
0807 08:34:43.579531 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":186,"threads_started":1}
I20260807 08:34:43.582581 15253 heartbeater.cc:347] Connected to a master server at 127.9.174.62:37305
I20260807 08:34:43.582971 15253 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:43.583796 15253 heartbeater.cc:511] Master 127.9.174.62:37305 requested a full tablet report, sending...
I20260807 08:34:43.586199 14974 ts_manager.cc:194] Registered new tserver with Master: f70be5334b9244f986e7b7dae6ac097e (127.9.174.1:34219)
I20260807 08:34:43.588356 14974 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:58895
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:34:43.603204 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37305
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:42989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:43.752655 15260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:43.753077 15260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:43.753172 15260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:43.758224 15260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:43.758358 15260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:43.758441 15260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:34:43.765687 15260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:42989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:43.767513 15260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:43.769136 15260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:43.782686 15269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:43.782696 15271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:43.782686 15268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:43.782932 15260 server_base.cc:1034] running on GCE node
I20260807 08:34:43.783645 15260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:43.784518 15260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:43.785745 15260 hybrid_clock.cc:648] HybridClock initialized: now 1786091683785701 us; error 64 us; skew 500 ppm
Aug 07 08:34:43 dist-test-slave-h3ft krb5kdc[14910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091683, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:43.789548 15260 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:34:43.790867 15260 webserver.cc:460] Webserver started at http://127.9.174.2:36607/ using document root <none> and password file <none>
I20260807 08:34:43.791673 15260 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:43.791781 15260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:43.792078 15260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:43.794739 15260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "621375f661b64edb81b5cd719dc0623e"
format_stamp: "Formatted at 2026-08-07 08:34:43 on dist-test-slave-h3ft"
server_key: "2253787c6cf3eab60cf739896051740d"
server_key_iv: "12cd84ff746756cd5b3215e60900fcc2"
server_key_version: "encryptionkey@0"
I20260807 08:34:43.795559 15260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "621375f661b64edb81b5cd719dc0623e"
format_stamp: "Formatted at 2026-08-07 08:34:43 on dist-test-slave-h3ft"
server_key: "2253787c6cf3eab60cf739896051740d"
server_key_iv: "12cd84ff746756cd5b3215e60900fcc2"
server_key_version: "encryptionkey@0"
I20260807 08:34:43.800513 15260 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 08:34:43.804164 15286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:43.805922 15260 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 08:34:43.806105 15260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "621375f661b64edb81b5cd719dc0623e"
format_stamp: "Formatted at 2026-08-07 08:34:43 on dist-test-slave-h3ft"
server_key: "2253787c6cf3eab60cf739896051740d"
server_key_iv: "12cd84ff746756cd5b3215e60900fcc2"
server_key_version: "encryptionkey@0"
I20260807 08:34:43.806265 15260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:43.832643 15260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:43.837366 15260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:43.837677 15260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:43.838541 15260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:43.839747 15260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:43.839831 15260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:43.839915 15260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:43.839960 15260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:43.852790 15260 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:37301
I20260807 08:34:43.852813 15436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:37301 every 8 connection(s)
I20260807 08:34:43.854092 15260 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:34:43.864356 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 15260
I20260807 08:34:43.864537 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:34:43.864884 9912 external_mini_cluster.cc:1428] Setting key 0879525646d9c09c26dd13a34a7b5e27
Aug 07 08:34:43 dist-test-slave-h3ft krb5kdc[14910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091683, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:43.873054 15033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:43.856877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:45527 (local address 127.9.174.62:37305)
0807 08:34:43.857119 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:43.857123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:43.858133 (+ 1010us) server_negotiation.cc:407] Connection header received
0807 08:34:43.859812 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:43.859815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:43.859878 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:43.860014 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:43.861450 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:43.862037 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:43.863430 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:43.863674 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:43.863773 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:43.868041 (+ 4268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:43.868044 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:43.868047 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:43.868074 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:43.869865 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:43.870578 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:43.870584 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:43.870586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:43.870699 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:43.871423 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:43.871426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:43.871428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:43.871700 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:43.871859 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:43.872601 (+ 742us) server_negotiation.cc:299] Negotiation successful
0807 08:34:43.872795 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 08:34:43.874302 15441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:43.857340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37305 (local address 127.9.174.2:45527)
0807 08:34:43.857922 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:43.857976 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:43.859535 (+ 1559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:43.860195 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:43.860203 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:43.860670 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:43.861298 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:43.861315 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:43.862163 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:43.862166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:43.863293 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:43.863304 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:43.863943 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:43.863950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:43.864254 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:43.865278 (+ 1024us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:43.865303 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:43.867784 (+ 2481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:43.870033 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:43.870040 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:43.870061 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:43.870445 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:43.870843 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:43.870847 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:43.870851 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:43.871254 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:43.871849 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:43.871859 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:43.872244 (+ 385us) client_negotiation.cc:769] Sending connection context
0807 08:34:43.872515 (+ 271us) client_negotiation.cc:240] Negotiation successful
0807 08:34:43.873022 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":141,"threads_started":1}
I20260807 08:34:43.876057 15437 heartbeater.cc:347] Connected to a master server at 127.9.174.62:37305
I20260807 08:34:43.876412 15437 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:43.877266 15437 heartbeater.cc:511] Master 127.9.174.62:37305 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
I20260807 08:34:43.878973 14974 ts_manager.cc:194] Registered new tserver with Master: 621375f661b64edb81b5cd719dc0623e (127.9.174.2:37301)
I20260807 08:34:43.879756 14974 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:45527
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:34:43.903400 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37305
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:42989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:44.051366 15445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:44.051787 15445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:44.051939 15445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:44.056841 15445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:44.056986 15445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:44.057096 15445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:34:44.063449 15445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:42989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:44.064962 15445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:44.066332 15445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:44.078948 15453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:44.078938 15456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:44.079025 15445 server_base.cc:1034] running on GCE node
W20260807 08:34:44.078971 15452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:44.079814 15445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:44.080655 15445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:44.081843 15445 hybrid_clock.cc:648] HybridClock initialized: now 1786091684081829 us; error 47 us; skew 500 ppm
Aug 07 08:34:44 dist-test-slave-h3ft krb5kdc[14910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091684, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:44.085718 15445 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:34:44.087080 15445 webserver.cc:460] Webserver started at http://127.9.174.3:35797/ using document root <none> and password file <none>
I20260807 08:34:44.087855 15445 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:44.087936 15445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:44.088156 15445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:44.090843 15445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "dcb66b97c51d4babbf22b72aa9c88874"
format_stamp: "Formatted at 2026-08-07 08:34:44 on dist-test-slave-h3ft"
server_key: "c841c93c463e7317bda1676d95ac8740"
server_key_iv: "89396768759ab116005b5d67e5e7141b"
server_key_version: "encryptionkey@0"
I20260807 08:34:44.091595 15445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "dcb66b97c51d4babbf22b72aa9c88874"
format_stamp: "Formatted at 2026-08-07 08:34:44 on dist-test-slave-h3ft"
server_key: "c841c93c463e7317bda1676d95ac8740"
server_key_iv: "89396768759ab116005b5d67e5e7141b"
server_key_version: "encryptionkey@0"
I20260807 08:34:44.096704 15445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 08:34:44.100605 15464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:44.102721 15445 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260807 08:34:44.102928 15445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "dcb66b97c51d4babbf22b72aa9c88874"
format_stamp: "Formatted at 2026-08-07 08:34:44 on dist-test-slave-h3ft"
server_key: "c841c93c463e7317bda1676d95ac8740"
server_key_iv: "89396768759ab116005b5d67e5e7141b"
server_key_version: "encryptionkey@0"
I20260807 08:34:44.103207 15445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:44.129278 15445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:44.133755 15445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:44.134028 15445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:44.134850 15445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:44.135999 15445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:44.136080 15445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:44.136168 15445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:44.136226 15445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:44.147897 15445 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:37773
I20260807 08:34:44.147918 15622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:37773 every 8 connection(s)
I20260807 08:34:44.149088 15445 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:34:44.153877 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 15445
I20260807 08:34:44.154043 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:34:44.154378 9912 external_mini_cluster.cc:1428] Setting key e26be3166c14593d978b4d47bf86ad6a
Aug 07 08:34:44 dist-test-slave-h3ft krb5kdc[14910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091684, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:44.166491 15033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:44.151601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:51107 (local address 127.9.174.62:37305)
0807 08:34:44.151794 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:44.151799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:44.152862 (+ 1063us) server_negotiation.cc:407] Connection header received
0807 08:34:44.154732 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:44.154735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:44.154806 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:44.154930 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:44.156463 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:44.157037 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:44.158385 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:44.158627 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:44.158747 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:44.161875 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:44.161878 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:44.161881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:44.161922 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:44.163939 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:44.164610 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:44.164619 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:44.164621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:44.164705 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:44.165133 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:44.165138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:44.165140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:44.165373 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:44.165496 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:44.166130 (+ 634us) server_negotiation.cc:299] Negotiation successful
0807 08:34:44.166291 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 08:34:44.167826 15625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:44.152072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37305 (local address 127.9.174.3:51107)
0807 08:34:44.152683 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:44.152735 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:44.154491 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:44.155182 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:44.155190 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:44.155637 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:44.156249 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:44.156265 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:44.157188 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:44.157192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:44.158218 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:44.158231 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:44.158827 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:44.158831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:44.159069 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:44.159778 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:44.159799 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:44.161627 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:44.164102 (+ 2475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:44.164110 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:44.164133 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:44.164480 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:44.164811 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:44.164814 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:44.164816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:44.165021 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:44.165495 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:44.165505 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:44.165852 (+ 347us) client_negotiation.cc:769] Sending connection context
0807 08:34:44.166090 (+ 238us) client_negotiation.cc:240] Negotiation successful
0807 08:34:44.166554 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":171,"threads_started":1}
I20260807 08:34:44.169396 15623 heartbeater.cc:347] Connected to a master server at 127.9.174.62:37305
I20260807 08:34:44.169723 15623 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:44.170368 15623 heartbeater.cc:511] Master 127.9.174.62:37305 requested a full tablet report, sending...
I20260807 08:34:44.171732 14974 ts_manager.cc:194] Registered new tserver with Master: dcb66b97c51d4babbf22b72aa9c88874 (127.9.174.3:37773)
I20260807 08:34:44.172485 14974 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:51107
I20260807 08:34:44.181345 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:34:44 dist-test-slave-h3ft krb5kdc[14910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:34:44 dist-test-slave-h3ft krb5kdc[14910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.1@KRBTEST.COM
I20260807 08:34:44.217244 15630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:44.205194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37642 (local address 127.9.174.1:34219)
0807 08:34:44.205597 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:44.205605 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:44.205642 (+ 37us) server_negotiation.cc:407] Connection header received
0807 08:34:44.205707 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:44.205713 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:44.205979 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:44.206149 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:44.206932 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:44.207945 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:44.208930 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:44.209376 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:44.209484 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:44.212632 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:44.212650 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:44.212669 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:44.212708 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:44.214957 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:44.215526 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:44.215533 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:44.215534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:44.215636 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:44.215998 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:44.216001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:44.216002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:44.216336 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:44.216589 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:44.216727 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 08:34:44.216952 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":120,"threads_started":1}
Aug 07 08:34:44 dist-test-slave-h3ft krb5kdc[14910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:44.230242 15033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:44.221203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49722 (local address 127.9.174.62:37305)
0807 08:34:44.221375 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:44.221380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:44.221397 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:34:44.221537 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:44.221540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:44.221589 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:44.221661 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:44.222368 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:44.222942 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:44.223801 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:44.224071 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:44.224166 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:44.226849 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:44.226852 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:44.226855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:44.226883 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:44.228353 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:44.228815 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:44.228823 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:44.228825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:44.228908 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:44.229200 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:44.229204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:44.229205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:44.229490 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:44.229655 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:44.229894 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 08:34:44.230040 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 08:34:44.230753 14974 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49722
I20260807 08:34:44.231575 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 15070
I20260807 08:34:44.262928 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 15260
I20260807 08:34:44.280784 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 15445
I20260807 08:34:44.299217 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 14931
2026-08-07T08:34:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2832 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:34:44 dist-test-slave-h3ft krb5kdc[15638](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:34:44 dist-test-slave-h3ft krb5kdc[15638](info): set up 2 sockets
Aug 07 08:34:44 dist-test-slave-h3ft krb5kdc[15638](info): commencing operation
krb5kdc: starting...
W20260807 08:34:45.607494 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.004s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:34:45 dist-test-slave-h3ft krb5kdc[15638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:34:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:34:45.769068 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:41981
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:41981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:45.913180 15658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:45.913582 15658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:45.913633 15658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:45.917969 15658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:45.918071 15658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:45.918092 15658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:45.918111 15658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:45.918128 15658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:45.924352 15658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:41981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:41981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:45.925799 15658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:45.927095 15658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:45.938289 15670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:45.938289 15668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:45.938289 15666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:45.938984 15658 server_base.cc:1034] running on GCE node
I20260807 08:34:45.939833 15658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:45.941358 15658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:45.942596 15658 hybrid_clock.cc:648] HybridClock initialized: now 1786091685942567 us; error 53 us; skew 500 ppm
Aug 07 08:34:45 dist-test-slave-h3ft krb5kdc[15638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091685, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:45.946323 15658 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:34:45.947510 15658 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:45.947618 15658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:45.947924 15658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:45.950834 15658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "42c5e2faf742400ab4d8a09a9e87ccbf"
format_stamp: "Formatted at 2026-08-07 08:34:45 on dist-test-slave-h3ft"
server_key: "64e0ea5012cd93c541fea81faae37505"
server_key_iv: "8bec7aef4c24003bbd9cb8483eaacc6b"
server_key_version: "encryptionkey@0"
I20260807 08:34:45.951701 15658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "42c5e2faf742400ab4d8a09a9e87ccbf"
format_stamp: "Formatted at 2026-08-07 08:34:45 on dist-test-slave-h3ft"
server_key: "64e0ea5012cd93c541fea81faae37505"
server_key_iv: "8bec7aef4c24003bbd9cb8483eaacc6b"
server_key_version: "encryptionkey@0"
I20260807 08:34:45.956703 15658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 08:34:45.959877 15680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:45.961755 15658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:34:45.961942 15658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "42c5e2faf742400ab4d8a09a9e87ccbf"
format_stamp: "Formatted at 2026-08-07 08:34:45 on dist-test-slave-h3ft"
server_key: "64e0ea5012cd93c541fea81faae37505"
server_key_iv: "8bec7aef4c24003bbd9cb8483eaacc6b"
server_key_version: "encryptionkey@0"
I20260807 08:34:45.962092 15658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:45.987740 15658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:45.992535 15658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:45.992846 15658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:46.002213 15658 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:41981
I20260807 08:34:46.002527 15751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:41981 every 8 connection(s)
I20260807 08:34:46.003209 15658 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:46.006863 15752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:46.010236 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 15658
I20260807 08:34:46.010435 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:34:46.010746 9912 external_mini_cluster.cc:1428] Setting key 4ecac07a38e7b9ef6bd4823580c95f2f
I20260807 08:34:46.014367 15752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf: Bootstrap starting.
I20260807 08:34:46.017274 15752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:46.018193 15752 log.cc:824] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:34:46 dist-test-slave-h3ft krb5kdc[15638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:46.021144 15752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf: No bootstrap required, opened a new log
I20260807 08:34:46.026214 15752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c5e2faf742400ab4d8a09a9e87ccbf" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 41981 } }
I20260807 08:34:46.026564 15752 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:46.026623 15752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42c5e2faf742400ab4d8a09a9e87ccbf, State: Initialized, Role: FOLLOWER
I20260807 08:34:46.027311 15752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c5e2faf742400ab4d8a09a9e87ccbf" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 41981 } }
I20260807 08:34:46.027513 15752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:34:46.027601 15752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:34:46.027709 15752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:46.028813 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:46.012552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56748 (local address 127.9.174.62:41981)
0807 08:34:46.013087 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:46.013105 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:46.013149 (+ 44us) server_negotiation.cc:407] Connection header received
0807 08:34:46.013865 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:46.013888 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:46.014286 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:46.014684 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:46.015661 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:46.016780 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:46.017756 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:46.018282 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:46.018473 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:46.021730 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:46.021737 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:46.021757 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:46.021793 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:46.024993 (+ 3200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:46.025524 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:46.025532 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:46.025538 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:46.025692 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:46.025996 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:46.025999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:46.026001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:46.026605 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:46.026862 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:46.027220 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 08:34:46.027587 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":161,"threads_started":1}
I20260807 08:34:46.029124 15752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c5e2faf742400ab4d8a09a9e87ccbf" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 41981 } }
I20260807 08:34:46.029556 15752 leader_election.cc:302] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42c5e2faf742400ab4d8a09a9e87ccbf; no voters:
I20260807 08:34:46.029982 15752 leader_election.cc:288] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:34:46.030124 15758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:46.030385 15758 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [term 1 LEADER]: Becoming Leader. State: Replica: 42c5e2faf742400ab4d8a09a9e87ccbf, State: Running, Role: LEADER
I20260807 08:34:46.030808 15758 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c5e2faf742400ab4d8a09a9e87ccbf" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 41981 } }
I20260807 08:34:46.031631 15752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:46.033106 15761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42c5e2faf742400ab4d8a09a9e87ccbf. Latest consensus state: current_term: 1 leader_uuid: "42c5e2faf742400ab4d8a09a9e87ccbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c5e2faf742400ab4d8a09a9e87ccbf" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 41981 } } }
I20260807 08:34:46.033106 15760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42c5e2faf742400ab4d8a09a9e87ccbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c5e2faf742400ab4d8a09a9e87ccbf" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 41981 } } }
I20260807 08:34:46.033319 15761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:46.033365 15760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:46.033787 15774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:46.037801 15774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:46.045491 15774 catalog_manager.cc:1347] Generated new cluster ID: f4cf0007c87e4ad9b4847099dccaf3ee
I20260807 08:34:46.045607 15774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:46.065707 15774 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:46.067365 15774 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:46.082305 15774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf: Generated new TSK 0
I20260807 08:34:46.083473 15774 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:34:46.135504 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:41981
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:46.274551 15789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:46.274911 15789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:46.274966 15789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:46.279285 15789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:46.279361 15789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:46.279430 15789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:34:46.285694 15789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:41981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:46.287237 15789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:46.288559 15789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:46.300966 15797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:46.301144 15789 server_base.cc:1034] running on GCE node
W20260807 08:34:46.301023 15801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:46.301254 15798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:46.302039 15789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:46.302937 15789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:46.304169 15789 hybrid_clock.cc:648] HybridClock initialized: now 1786091686304129 us; error 55 us; skew 500 ppm
Aug 07 08:34:46 dist-test-slave-h3ft krb5kdc[15638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091686, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:46.307731 15789 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:34:46.308795 15789 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:46.308877 15789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:46.309146 15789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:46.312047 15789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "e26a88c37cbc4082b86158d99a37d06f"
format_stamp: "Formatted at 2026-08-07 08:34:46 on dist-test-slave-h3ft"
server_key: "0db52f5396d5cf7a494b9323f2f9efd7"
server_key_iv: "7a41be94cad8647ee43cb464de90c41d"
server_key_version: "encryptionkey@0"
I20260807 08:34:46.312788 15789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "e26a88c37cbc4082b86158d99a37d06f"
format_stamp: "Formatted at 2026-08-07 08:34:46 on dist-test-slave-h3ft"
server_key: "0db52f5396d5cf7a494b9323f2f9efd7"
server_key_iv: "7a41be94cad8647ee43cb464de90c41d"
server_key_version: "encryptionkey@0"
I20260807 08:34:46.317709 15789 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:34:46.320840 15809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:46.322480 15789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:34:46.322640 15789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "e26a88c37cbc4082b86158d99a37d06f"
format_stamp: "Formatted at 2026-08-07 08:34:46 on dist-test-slave-h3ft"
server_key: "0db52f5396d5cf7a494b9323f2f9efd7"
server_key_iv: "7a41be94cad8647ee43cb464de90c41d"
server_key_version: "encryptionkey@0"
I20260807 08:34:46.322782 15789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:46.351496 15789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:46.356238 15789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:46.356556 15789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:46.357447 15789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:46.358652 15789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:46.358712 15789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:46.358803 15789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:46.358850 15789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:46.371423 15789 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:42981
I20260807 08:34:46.371440 15972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:42981 every 8 connection(s)
I20260807 08:34:46.372248 15789 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:34:46.376161 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 15789
I20260807 08:34:46.376332 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:34:46.376665 9912 external_mini_cluster.cc:1428] Setting key 279f0579bcffe5506361b909d8d3c5fd
Aug 07 08:34:46 dist-test-slave-h3ft krb5kdc[15638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091686, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
I20260807 08:34:46.390347 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:46.374581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:34823 (local address 127.9.174.62:41981)
0807 08:34:46.374802 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:46.374807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:46.375824 (+ 1017us) server_negotiation.cc:407] Connection header received
0807 08:34:46.377510 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:46.377514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:46.377589 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:46.377698 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:46.379157 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:46.379741 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:46.381267 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:46.381524 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:46.381617 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:46.385263 (+ 3646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:46.385268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:46.385271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:46.385314 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:46.387408 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:46.388149 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:46.388154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:46.388156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:46.388269 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:46.388746 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:46.388752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:46.388756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:46.389060 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:46.389186 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:46.389896 (+ 710us) server_negotiation.cc:299] Negotiation successful
0807 08:34:46.390077 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 08:34:46.391522 15977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:46.375076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:41981 (local address 127.9.174.1:34823)
0807 08:34:46.375633 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:46.375682 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:46.377250 (+ 1568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:46.377904 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:46.377911 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:46.378344 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:46.378915 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:46.378930 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:46.379882 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:46.379886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:46.381117 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:46.381132 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:46.381711 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:46.381716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:46.381959 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:46.382726 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:46.382755 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:46.385023 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:46.387586 (+ 2563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:46.387593 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:46.387612 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:46.388004 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:46.388407 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:46.388410 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:46.388412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:46.388605 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:46.389202 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:46.389209 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:46.389581 (+ 372us) client_negotiation.cc:769] Sending connection context
0807 08:34:46.389778 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 08:34:46.390244 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":134,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
I20260807 08:34:46.393276 15974 heartbeater.cc:347] Connected to a master server at 127.9.174.62:41981
I20260807 08:34:46.393635 15974 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:46.394410 15974 heartbeater.cc:511] Master 127.9.174.62:41981 requested a full tablet report, sending...
I20260807 08:34:46.396735 15706 ts_manager.cc:194] Registered new tserver with Master: e26a88c37cbc4082b86158d99a37d06f (127.9.174.1:42981)
I20260807 08:34:46.398144 15706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:34823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:34:46.415396 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:41981
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:46.564965 15980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:46.565330 15980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:46.565420 15980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:46.570307 15980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:46.570416 15980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:46.570525 15980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:34:46.576963 15980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:41981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:46.578459 15980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:46.579908 15980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:46.592295 15986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:46.592295 15992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:46.592325 15988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:46.592528 15980 server_base.cc:1034] running on GCE node
I20260807 08:34:46.593281 15980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:46.594080 15980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:46.595243 15980 hybrid_clock.cc:648] HybridClock initialized: now 1786091686595214 us; error 42 us; skew 500 ppm
Aug 07 08:34:46 dist-test-slave-h3ft krb5kdc[15638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091686, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:46.598775 15980 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:34:46.599840 15980 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:46.599916 15980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:46.600222 15980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:46.603044 15980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "4e19ea094b544f08a9a37eac7ef7d353"
format_stamp: "Formatted at 2026-08-07 08:34:46 on dist-test-slave-h3ft"
server_key: "2a1b27b38e2a0f725044de7b97d672be"
server_key_iv: "e161a5a5f6bf0b746c9d8a1712f60c6b"
server_key_version: "encryptionkey@0"
I20260807 08:34:46.603751 15980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "4e19ea094b544f08a9a37eac7ef7d353"
format_stamp: "Formatted at 2026-08-07 08:34:46 on dist-test-slave-h3ft"
server_key: "2a1b27b38e2a0f725044de7b97d672be"
server_key_iv: "e161a5a5f6bf0b746c9d8a1712f60c6b"
server_key_version: "encryptionkey@0"
I20260807 08:34:46.608484 15980 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 08:34:46.611559 16001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:46.613269 15980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:34:46.613425 15980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "4e19ea094b544f08a9a37eac7ef7d353"
format_stamp: "Formatted at 2026-08-07 08:34:46 on dist-test-slave-h3ft"
server_key: "2a1b27b38e2a0f725044de7b97d672be"
server_key_iv: "e161a5a5f6bf0b746c9d8a1712f60c6b"
server_key_version: "encryptionkey@0"
I20260807 08:34:46.613569 15980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:46.654776 15980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:46.658695 15980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:46.658946 15980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:46.659725 15980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:46.660765 15980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:46.660859 15980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:46.660966 15980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:46.661021 15980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:46.672860 15980 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:40931
I20260807 08:34:46.672880 16165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:40931 every 8 connection(s)
I20260807 08:34:46.673671 15980 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:34:46.676002 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 15980
I20260807 08:34:46.676137 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:34:46.676519 9912 external_mini_cluster.cc:1428] Setting key 00310d99a40025587a6ef451bdfc5894
Aug 07 08:34:46 dist-test-slave-h3ft krb5kdc[15638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091686, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
I20260807 08:34:46.690151 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:46.676055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:46065 (local address 127.9.174.62:41981)
0807 08:34:46.676277 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:46.676281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:46.677451 (+ 1170us) server_negotiation.cc:407] Connection header received
0807 08:34:46.679085 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:46.679088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:46.679152 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:46.679264 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:46.680690 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:46.681296 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:46.682589 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:46.682833 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:46.682958 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:46.685956 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:46.685959 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:46.685962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:46.685989 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:46.687617 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:46.688200 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:46.688207 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:46.688209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:46.688304 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:46.688702 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:46.688706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:46.688708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:46.688971 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:46.689090 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:46.689739 (+ 649us) server_negotiation.cc:299] Negotiation successful
0807 08:34:46.689895 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260807 08:34:46.691342 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:46.676661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:41981 (local address 127.9.174.2:46065)
0807 08:34:46.677272 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:46.677335 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:46.678811 (+ 1476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:46.679476 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:46.679483 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:46.679913 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:46.680519 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:46.680536 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:46.681450 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:46.681453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:46.682442 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:46.682452 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:46.683041 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:46.683047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:46.683245 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:46.683976 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:46.683995 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:46.685760 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:46.687757 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:46.687764 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:46.687784 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:46.688086 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:46.688410 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:46.688414 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:46.688416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:46.688594 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:46.689105 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:46.689114 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:46.689429 (+ 315us) client_negotiation.cc:769] Sending connection context
0807 08:34:46.689692 (+ 263us) client_negotiation.cc:240] Negotiation successful
0807 08:34:46.690109 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":176,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
I20260807 08:34:46.693054 16166 heartbeater.cc:347] Connected to a master server at 127.9.174.62:41981
I20260807 08:34:46.693390 16166 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:46.694046 16166 heartbeater.cc:511] Master 127.9.174.62:41981 requested a full tablet report, sending...
I20260807 08:34:46.695575 15706 ts_manager.cc:194] Registered new tserver with Master: 4e19ea094b544f08a9a37eac7ef7d353 (127.9.174.2:40931)
I20260807 08:34:46.696296 15706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:46065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:34:46.711453 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:41981
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:46.851576 16176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:46.852039 16176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:46.852154 16176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:46.856824 16176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:46.856911 16176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:46.857014 16176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:34:46.863127 16176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:41981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:46.864566 16176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:46.865886 16176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:46.877864 16188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:46.877918 16191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:46.878185 16176 server_base.cc:1034] running on GCE node
W20260807 08:34:46.877864 16189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:46.878957 16176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:46.879814 16176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:46.881023 16176 hybrid_clock.cc:648] HybridClock initialized: now 1786091686880985 us; error 46 us; skew 500 ppm
Aug 07 08:34:46 dist-test-slave-h3ft krb5kdc[15638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091686, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:46.884712 16176 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:34:46.885703 16176 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:46.885816 16176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:46.886129 16176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:46.888849 16176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "e4dcc489eaa1468fa11011e1f700a8dc"
format_stamp: "Formatted at 2026-08-07 08:34:46 on dist-test-slave-h3ft"
server_key: "4d813bd954ce19772a9486074a87e2e1"
server_key_iv: "df3d184cabca11fb2d58fbe926362fe7"
server_key_version: "encryptionkey@0"
I20260807 08:34:46.889619 16176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "e4dcc489eaa1468fa11011e1f700a8dc"
format_stamp: "Formatted at 2026-08-07 08:34:46 on dist-test-slave-h3ft"
server_key: "4d813bd954ce19772a9486074a87e2e1"
server_key_iv: "df3d184cabca11fb2d58fbe926362fe7"
server_key_version: "encryptionkey@0"
I20260807 08:34:46.894661 16176 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:34:46.897716 16198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:46.899312 16176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:34:46.899472 16176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "e4dcc489eaa1468fa11011e1f700a8dc"
format_stamp: "Formatted at 2026-08-07 08:34:46 on dist-test-slave-h3ft"
server_key: "4d813bd954ce19772a9486074a87e2e1"
server_key_iv: "df3d184cabca11fb2d58fbe926362fe7"
server_key_version: "encryptionkey@0"
I20260807 08:34:46.899608 16176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:46.917906 16176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:46.921845 16176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:46.922096 16176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:46.922840 16176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:46.923938 16176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:46.924015 16176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:46.924095 16176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:46.924147 16176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:46.935948 16176 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:46327
I20260807 08:34:46.935969 16353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:46327 every 8 connection(s)
I20260807 08:34:46.936750 16176 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:34:46.940811 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 16176
I20260807 08:34:46.940984 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:34:46.941330 9912 external_mini_cluster.cc:1428] Setting key 67ab11f37ee4335d00beac2d60adc8cb
Aug 07 08:34:46 dist-test-slave-h3ft krb5kdc[15638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091686, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:46.953119 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:46.939111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:52325 (local address 127.9.174.62:41981)
0807 08:34:46.939297 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:46.939302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:46.940587 (+ 1285us) server_negotiation.cc:407] Connection header received
0807 08:34:46.942324 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:46.942328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:46.942399 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:46.942527 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:46.944203 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:46.944805 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:46.946076 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:46.946317 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:46.946430 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:46.949345 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:46.949349 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:46.949351 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:46.949377 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:46.950703 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:46.951347 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:46.951350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:46.951351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:46.951430 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:46.951846 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:46.951848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:46.951850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:46.952053 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:46.952168 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:46.952763 (+ 595us) server_negotiation.cc:299] Negotiation successful
0807 08:34:46.952912 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 08:34:46.954151 16356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:46.939595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:41981 (local address 127.9.174.3:52325)
0807 08:34:46.940377 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:46.940433 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:46.942073 (+ 1640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:46.942767 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:46.942775 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:46.943317 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:46.944046 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:46.944065 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:46.944955 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:46.944958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:46.945933 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:46.945943 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:46.946499 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:46.946504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:46.946703 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:46.947408 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:46.947426 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:46.949144 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:46.950837 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:46.950843 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:46.950861 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:46.951228 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:46.951550 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:46.951553 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:46.951555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:46.951718 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:46.952168 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:46.952177 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:46.952484 (+ 307us) client_negotiation.cc:769] Sending connection context
0807 08:34:46.952706 (+ 222us) client_negotiation.cc:240] Negotiation successful
0807 08:34:46.953110 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":164,"threads_started":1}
I20260807 08:34:46.955577 16354 heartbeater.cc:347] Connected to a master server at 127.9.174.62:41981
I20260807 08:34:46.955909 16354 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:46.956542 16354 heartbeater.cc:511] Master 127.9.174.62:41981 requested a full tablet report, sending...
I20260807 08:34:46.957751 15705 ts_manager.cc:194] Registered new tserver with Master: e4dcc489eaa1468fa11011e1f700a8dc (127.9.174.3:46327)
I20260807 08:34:46.958420 15705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:52325
I20260807 08:34:46.968333 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:34:46.979295 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:46.970856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56750 (local address 127.9.174.62:41981)
0807 08:34:46.971110 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:46.971114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:46.971254 (+ 140us) server_negotiation.cc:407] Connection header received
0807 08:34:46.971366 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:46.971368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:46.971424 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:46.971502 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:46.972292 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:46.972913 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:46.973883 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:46.974172 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:46.974265 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:46.975845 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:46.975848 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:46.975851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:46.975875 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:46.977479 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:46.977945 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:46.977948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:46.977950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:46.978033 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:46.978309 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:46.978312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:46.978313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:46.978571 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:46.978692 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:46.978889 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 08:34:46.979067 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 08:34:46.982808 15705 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:34:46.986691 15705 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:34:47.005316 16372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:46.997515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:46327 (local address 127.0.0.1:36218)
0807 08:34:46.998231 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:46.998295 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:46.998444 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:47.000029 (+ 1585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:47.000032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:47.000070 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:47.000191 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:47.000199 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.001423 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.001430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:47.003175 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:47.003183 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.004634 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.004638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:47.004775 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:47.004829 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 08:34:47.004926 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 08:34:47.005086 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":214,"spinlock_wait_cycles":14080,"thread_start_us":130,"threads_started":1}
I20260807 08:34:47.005342 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:46.997238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:40931 (local address 127.0.0.1:34414)
0807 08:34:46.998232 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:46.998295 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:46.998445 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:46.999481 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:46.999484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:46.999505 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:46.999847 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:46.999857 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.001697 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.001704 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:47.003206 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:47.003219 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.004845 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.004849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:47.004999 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:47.005028 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 08:34:47.005082 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 08:34:47.005159 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"mutex_wait_us":149,"spinlock_wait_cycles":4736,"thread_start_us":136,"threads_started":1}
I20260807 08:34:47.005707 16374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:46.997696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:42981 (local address 127.0.0.1:45708)
0807 08:34:46.998709 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:46.998723 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:46.998846 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:46.999454 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:46.999464 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:46.999495 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:46.999847 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:46.999857 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.001649 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.001652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:47.004010 (+ 2358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:47.004021 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.005284 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.005287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:47.005387 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:47.005416 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 08:34:47.005483 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 08:34:47.005561 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"mutex_wait_us":1057,"spinlock_wait_cycles":1408,"thread_start_us":90,"threads_started":1}
I20260807 08:34:47.006090 16378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:46.998153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34414 (local address 127.9.174.2:40931)
0807 08:34:46.998829 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:46.998843 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:46.998869 (+ 26us) server_negotiation.cc:407] Connection header received
0807 08:34:46.998966 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:46.998972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:46.999262 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:47.000008 (+ 746us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:47.000258 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.001514 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.003429 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.004687 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.005470 (+ 783us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:47.005661 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:47.005722 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 08:34:47.005892 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":158,"threads_started":1}
I20260807 08:34:47.006474 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:46.998833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36218 (local address 127.9.174.3:46327)
0807 08:34:46.999360 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:46.999377 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:46.999403 (+ 26us) server_negotiation.cc:407] Connection header received
0807 08:34:46.999501 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:46.999660 (+ 159us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:46.999865 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:47.000382 (+ 517us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:47.000547 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.001286 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.003322 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.004493 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.005918 (+ 1425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:47.006128 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:47.006188 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 08:34:47.006297 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":244,"threads_started":1}
I20260807 08:34:47.006850 16376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:46.997975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45708 (local address 127.9.174.1:42981)
0807 08:34:46.998468 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:46.998478 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:46.998846 (+ 368us) server_negotiation.cc:407] Connection header received
0807 08:34:46.998964 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:46.998974 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:46.999264 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:46.999444 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:47.000022 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.001507 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.004259 (+ 2752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.005140 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.006233 (+ 1093us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:47.006389 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:47.006445 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 08:34:47.006658 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":102,"threads_started":1}
I20260807 08:34:47.010131 15879 tablet_service.cc:1467] Processing CreateTablet for tablet f50b4b97152d4509b129f4144adde7fd (DEFAULT_TABLE table=test-table [id=4a1ef8d63252498c97a2f1c0caf8828b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:47.010172 16067 tablet_service.cc:1467] Processing CreateTablet for tablet f50b4b97152d4509b129f4144adde7fd (DEFAULT_TABLE table=test-table [id=4a1ef8d63252498c97a2f1c0caf8828b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:47.011483 16067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f50b4b97152d4509b129f4144adde7fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:47.011487 15879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f50b4b97152d4509b129f4144adde7fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:47.013161 16259 tablet_service.cc:1467] Processing CreateTablet for tablet f50b4b97152d4509b129f4144adde7fd (DEFAULT_TABLE table=test-table [id=4a1ef8d63252498c97a2f1c0caf8828b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:47.014456 16259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f50b4b97152d4509b129f4144adde7fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:47.021508 16382 tablet_bootstrap.cc:492] T f50b4b97152d4509b129f4144adde7fd P e26a88c37cbc4082b86158d99a37d06f: Bootstrap starting.
I20260807 08:34:47.024147 16386 tablet_bootstrap.cc:492] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc: Bootstrap starting.
I20260807 08:34:47.024362 16382 tablet_bootstrap.cc:654] T f50b4b97152d4509b129f4144adde7fd P e26a88c37cbc4082b86158d99a37d06f: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:47.024595 16385 tablet_bootstrap.cc:492] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353: Bootstrap starting.
I20260807 08:34:47.025537 16382 log.cc:824] T f50b4b97152d4509b129f4144adde7fd P e26a88c37cbc4082b86158d99a37d06f: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:47.026811 16386 tablet_bootstrap.cc:654] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:47.028072 16385 tablet_bootstrap.cc:654] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:47.028378 16386 log.cc:824] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:47.028445 16382 tablet_bootstrap.cc:492] T f50b4b97152d4509b129f4144adde7fd P e26a88c37cbc4082b86158d99a37d06f: No bootstrap required, opened a new log
I20260807 08:34:47.028685 16382 ts_tablet_manager.cc:1397] T f50b4b97152d4509b129f4144adde7fd P e26a88c37cbc4082b86158d99a37d06f: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 08:34:47.029085 16385 log.cc:824] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:47.030754 16386 tablet_bootstrap.cc:492] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc: No bootstrap required, opened a new log
I20260807 08:34:47.031087 16386 ts_tablet_manager.cc:1397] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260807 08:34:47.031414 16385 tablet_bootstrap.cc:492] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353: No bootstrap required, opened a new log
I20260807 08:34:47.031633 16385 ts_tablet_manager.cc:1397] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 08:34:47.034379 16382 raft_consensus.cc:348] T f50b4b97152d4509b129f4144adde7fd P e26a88c37cbc4082b86158d99a37d06f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e19ea094b544f08a9a37eac7ef7d353" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 40931 } } peers { permanent_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46327 } } peers { permanent_uuid: "e26a88c37cbc4082b86158d99a37d06f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42981 } }
I20260807 08:34:47.034721 16382 raft_consensus.cc:374] T f50b4b97152d4509b129f4144adde7fd P e26a88c37cbc4082b86158d99a37d06f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:47.034788 16382 raft_consensus.cc:729] T f50b4b97152d4509b129f4144adde7fd P e26a88c37cbc4082b86158d99a37d06f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e26a88c37cbc4082b86158d99a37d06f, State: Initialized, Role: FOLLOWER
I20260807 08:34:47.035393 16382 consensus_queue.cc:260] T f50b4b97152d4509b129f4144adde7fd P e26a88c37cbc4082b86158d99a37d06f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e19ea094b544f08a9a37eac7ef7d353" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 40931 } } peers { permanent_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46327 } } peers { permanent_uuid: "e26a88c37cbc4082b86158d99a37d06f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42981 } }
I20260807 08:34:47.036314 15974 heartbeater.cc:503] Master 127.9.174.62:41981 was elected leader, sending a full tablet report...
I20260807 08:34:47.036334 16385 raft_consensus.cc:348] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e19ea094b544f08a9a37eac7ef7d353" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 40931 } } peers { permanent_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46327 } } peers { permanent_uuid: "e26a88c37cbc4082b86158d99a37d06f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42981 } }
I20260807 08:34:47.036641 16385 raft_consensus.cc:374] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:47.036708 16385 raft_consensus.cc:729] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e19ea094b544f08a9a37eac7ef7d353, State: Initialized, Role: FOLLOWER
I20260807 08:34:47.036746 16382 ts_tablet_manager.cc:1428] T f50b4b97152d4509b129f4144adde7fd P e26a88c37cbc4082b86158d99a37d06f: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260807 08:34:47.037179 16385 consensus_queue.cc:260] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e19ea094b544f08a9a37eac7ef7d353" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 40931 } } peers { permanent_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46327 } } peers { permanent_uuid: "e26a88c37cbc4082b86158d99a37d06f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42981 } }
I20260807 08:34:47.038008 16166 heartbeater.cc:503] Master 127.9.174.62:41981 was elected leader, sending a full tablet report...
I20260807 08:34:47.036015 16386 raft_consensus.cc:348] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e19ea094b544f08a9a37eac7ef7d353" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 40931 } } peers { permanent_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46327 } } peers { permanent_uuid: "e26a88c37cbc4082b86158d99a37d06f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42981 } }
I20260807 08:34:47.038411 16386 raft_consensus.cc:374] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:47.038504 16386 raft_consensus.cc:729] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4dcc489eaa1468fa11011e1f700a8dc, State: Initialized, Role: FOLLOWER
I20260807 08:34:47.039085 16386 consensus_queue.cc:260] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e19ea094b544f08a9a37eac7ef7d353" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 40931 } } peers { permanent_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46327 } } peers { permanent_uuid: "e26a88c37cbc4082b86158d99a37d06f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42981 } }
I20260807 08:34:47.040349 16354 heartbeater.cc:503] Master 127.9.174.62:41981 was elected leader, sending a full tablet report...
I20260807 08:34:47.040683 16386 ts_tablet_manager.cc:1428] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc: Time spent starting tablet: real 0.010s user 0.003s sys 0.003s
I20260807 08:34:47.041462 16385 ts_tablet_manager.cc:1428] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260807 08:34:47.116905 16394 raft_consensus.cc:482] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:34:47.117348 16394 raft_consensus.cc:504] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e19ea094b544f08a9a37eac7ef7d353" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 40931 } } peers { permanent_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46327 } } peers { permanent_uuid: "e26a88c37cbc4082b86158d99a37d06f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42981 } }
I20260807 08:34:47.118649 16393 raft_consensus.cc:482] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:34:47.118897 16393 raft_consensus.cc:504] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e19ea094b544f08a9a37eac7ef7d353" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 40931 } } peers { permanent_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46327 } } peers { permanent_uuid: "e26a88c37cbc4082b86158d99a37d06f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42981 } }
I20260807 08:34:47.119349 16394 leader_election.cc:288] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e19ea094b544f08a9a37eac7ef7d353 (127.9.174.2:40931), e26a88c37cbc4082b86158d99a37d06f (127.9.174.1:42981)
I20260807 08:34:47.120402 16393 leader_election.cc:288] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4dcc489eaa1468fa11011e1f700a8dc (127.9.174.3:46327), e26a88c37cbc4082b86158d99a37d06f (127.9.174.1:42981)
W20260807 08:34:47.124138 15975 tablet.cc:2369] T f50b4b97152d4509b129f4144adde7fd P e26a88c37cbc4082b86158d99a37d06f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:34:47.126086 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:47.120580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:46327 (local address 127.9.174.2:55273)
0807 08:34:47.121036 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:47.121050 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:47.121141 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:47.121448 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:47.121451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:47.121471 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:47.121603 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:47.121609 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.123476 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.123479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:47.124741 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:47.124755 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.125661 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.125668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:47.125785 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:47.125821 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 08:34:47.125868 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:34:47.125938 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379}
I20260807 08:34:47.126096 16376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:47.119930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:60517 (local address 127.9.174.1:42981)
0807 08:34:47.120118 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:47.120124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:47.120659 (+ 535us) server_negotiation.cc:407] Connection header received
0807 08:34:47.120864 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:47.120869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:47.120935 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:47.120986 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:47.121430 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.122327 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.124836 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.125454 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.125563 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:47.125652 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:47.125855 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 08:34:47.125953 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 08:34:47.126116 16396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:47.119677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:42981 (local address 127.9.174.3:60517)
0807 08:34:47.120567 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:47.120581 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:47.120722 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:47.121138 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:47.121140 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:47.121157 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:47.121274 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:47.121279 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.122464 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.122467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:47.124684 (+ 2217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:47.124695 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.125593 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.125599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:47.125702 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:47.125736 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 08:34:47.125818 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:34:47.125908 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":1400,"thread_start_us":149,"threads_started":1}
I20260807 08:34:47.126749 16356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:47.119673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:40931 (local address 127.9.174.3:44863)
0807 08:34:47.119827 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:47.119848 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:47.119975 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:47.120566 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:47.120570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:47.120598 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:47.120798 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:47.120808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.122464 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.122467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:47.124950 (+ 2483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:47.124959 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.126258 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.126263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:47.126412 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:47.126446 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 08:34:47.126499 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 08:34:47.126589 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":1044}
I20260807 08:34:47.127096 15903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f50b4b97152d4509b129f4144adde7fd" candidate_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e26a88c37cbc4082b86158d99a37d06f" is_pre_election: true
I20260807 08:34:47.127292 16378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:47.120114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:44863 (local address 127.9.174.2:40931)
0807 08:34:47.120273 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:47.120278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:47.120295 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:34:47.120357 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:47.120360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:47.120422 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:47.120932 (+ 510us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:47.121056 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.122314 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.125090 (+ 2776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.126107 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.126970 (+ 863us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:47.127052 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:47.127094 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 08:34:47.127167 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":60}
I20260807 08:34:47.127557 15903 raft_consensus.cc:2454] T f50b4b97152d4509b129f4144adde7fd P e26a88c37cbc4082b86158d99a37d06f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4dcc489eaa1468fa11011e1f700a8dc in term 0.
I20260807 08:34:47.127627 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:47.120712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:42981 (local address 127.9.174.2:41717)
0807 08:34:47.121517 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:47.121531 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:47.121612 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:47.122826 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:47.122832 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:47.122856 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:47.123125 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:47.123134 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.124328 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.124331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:47.125582 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:47.125591 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.127211 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.127215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:47.127311 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:47.127333 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:34:47.127395 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 08:34:47.127462 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":22,"thread_start_us":123,"threads_started":1}
I20260807 08:34:47.127997 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:47.120943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:55273 (local address 127.9.174.3:46327)
0807 08:34:47.121069 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:47.121075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:47.121092 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:34:47.121252 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:47.121255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:47.121307 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:47.122509 (+ 1202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:47.122590 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.123346 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.124954 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.125500 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.127203 (+ 1703us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:47.127501 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:47.127542 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 08:34:47.127615 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 08:34:47.127976 16096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f50b4b97152d4509b129f4144adde7fd" candidate_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e19ea094b544f08a9a37eac7ef7d353" is_pre_election: true
I20260807 08:34:47.128208 16096 raft_consensus.cc:2454] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4dcc489eaa1468fa11011e1f700a8dc in term 0.
I20260807 08:34:47.128276 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:47.121395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:41717 (local address 127.9.174.1:42981)
0807 08:34:47.122336 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:47.122369 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:47.122391 (+ 22us) server_negotiation.cc:407] Connection header received
0807 08:34:47.122484 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:47.122487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:47.122580 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:47.123239 (+ 659us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:47.123343 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.124196 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.126286 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.126994 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.127935 (+ 941us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:47.127997 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:47.128038 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 08:34:47.128125 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":819,"thread_start_us":154,"threads_started":1}
I20260807 08:34:47.128237 16205 leader_election.cc:302] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e26a88c37cbc4082b86158d99a37d06f, e4dcc489eaa1468fa11011e1f700a8dc; no voters:
I20260807 08:34:47.128628 16394 raft_consensus.cc:2781] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:34:47.128782 16394 raft_consensus.cc:482] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:34:47.128624 16283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f50b4b97152d4509b129f4144adde7fd" candidate_uuid: "4e19ea094b544f08a9a37eac7ef7d353" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" is_pre_election: true
I20260807 08:34:47.128993 15903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f50b4b97152d4509b129f4144adde7fd" candidate_uuid: "4e19ea094b544f08a9a37eac7ef7d353" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e26a88c37cbc4082b86158d99a37d06f" is_pre_election: true
I20260807 08:34:47.129114 15903 raft_consensus.cc:2454] T f50b4b97152d4509b129f4144adde7fd P e26a88c37cbc4082b86158d99a37d06f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e19ea094b544f08a9a37eac7ef7d353 in term 0.
I20260807 08:34:47.128847 16394 raft_consensus.cc:3037] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:47.129563 16006 leader_election.cc:302] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e19ea094b544f08a9a37eac7ef7d353, e26a88c37cbc4082b86158d99a37d06f; no voters:
I20260807 08:34:47.129930 16393 raft_consensus.cc:2781] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:34:47.130033 16393 raft_consensus.cc:482] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:34:47.130074 16393 raft_consensus.cc:3037] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:47.131161 16394 raft_consensus.cc:504] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e19ea094b544f08a9a37eac7ef7d353" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 40931 } } peers { permanent_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46327 } } peers { permanent_uuid: "e26a88c37cbc4082b86158d99a37d06f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42981 } }
I20260807 08:34:47.131515 16283 raft_consensus.cc:2379] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e19ea094b544f08a9a37eac7ef7d353 in current term 1: Already voted for candidate e4dcc489eaa1468fa11011e1f700a8dc in this term.
I20260807 08:34:47.131644 16393 raft_consensus.cc:504] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e19ea094b544f08a9a37eac7ef7d353" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 40931 } } peers { permanent_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46327 } } peers { permanent_uuid: "e26a88c37cbc4082b86158d99a37d06f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42981 } }
I20260807 08:34:47.132085 16393 leader_election.cc:288] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [CANDIDATE]: Term 1 election: Requested vote from peers e4dcc489eaa1468fa11011e1f700a8dc (127.9.174.3:46327), e26a88c37cbc4082b86158d99a37d06f (127.9.174.1:42981)
I20260807 08:34:47.132261 15903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f50b4b97152d4509b129f4144adde7fd" candidate_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e26a88c37cbc4082b86158d99a37d06f"
I20260807 08:34:47.132401 15903 raft_consensus.cc:3037] T f50b4b97152d4509b129f4144adde7fd P e26a88c37cbc4082b86158d99a37d06f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:47.132483 16096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f50b4b97152d4509b129f4144adde7fd" candidate_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e19ea094b544f08a9a37eac7ef7d353"
I20260807 08:34:47.132690 16096 raft_consensus.cc:2379] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e4dcc489eaa1468fa11011e1f700a8dc in current term 1: Already voted for candidate 4e19ea094b544f08a9a37eac7ef7d353 in this term.
I20260807 08:34:47.133153 15901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f50b4b97152d4509b129f4144adde7fd" candidate_uuid: "4e19ea094b544f08a9a37eac7ef7d353" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e26a88c37cbc4082b86158d99a37d06f"
I20260807 08:34:47.133932 16394 leader_election.cc:288] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [CANDIDATE]: Term 1 election: Requested vote from peers 4e19ea094b544f08a9a37eac7ef7d353 (127.9.174.2:40931), e26a88c37cbc4082b86158d99a37d06f (127.9.174.1:42981)
I20260807 08:34:47.134027 15903 raft_consensus.cc:2454] T f50b4b97152d4509b129f4144adde7fd P e26a88c37cbc4082b86158d99a37d06f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4dcc489eaa1468fa11011e1f700a8dc in term 1.
I20260807 08:34:47.134033 16283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f50b4b97152d4509b129f4144adde7fd" candidate_uuid: "4e19ea094b544f08a9a37eac7ef7d353" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4dcc489eaa1468fa11011e1f700a8dc"
I20260807 08:34:47.134208 16283 raft_consensus.cc:2379] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e19ea094b544f08a9a37eac7ef7d353 in current term 1: Already voted for candidate e4dcc489eaa1468fa11011e1f700a8dc in this term.
I20260807 08:34:47.134341 16205 leader_election.cc:302] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e26a88c37cbc4082b86158d99a37d06f, e4dcc489eaa1468fa11011e1f700a8dc; no voters: 4e19ea094b544f08a9a37eac7ef7d353
I20260807 08:34:47.134536 16394 raft_consensus.cc:2781] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:47.134528 16004 leader_election.cc:302] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e19ea094b544f08a9a37eac7ef7d353; no voters: e26a88c37cbc4082b86158d99a37d06f, e4dcc489eaa1468fa11011e1f700a8dc
I20260807 08:34:47.134755 16393 raft_consensus.cc:2726] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 08:34:47.134821 16394 raft_consensus.cc:686] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [term 1 LEADER]: Becoming Leader. State: Replica: e4dcc489eaa1468fa11011e1f700a8dc, State: Running, Role: LEADER
I20260807 08:34:47.135329 16394 consensus_queue.cc:237] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e19ea094b544f08a9a37eac7ef7d353" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 40931 } } peers { permanent_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46327 } } peers { permanent_uuid: "e26a88c37cbc4082b86158d99a37d06f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42981 } }
I20260807 08:34:47.139539 15700 catalog_manager.cc:5495] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc reported cstate change: term changed from 0 to 1, leader changed from <none> to e4dcc489eaa1468fa11011e1f700a8dc (127.9.174.3). New cstate: current_term: 1 leader_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e19ea094b544f08a9a37eac7ef7d353" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 40931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4dcc489eaa1468fa11011e1f700a8dc" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e26a88c37cbc4082b86158d99a37d06f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42981 } health_report { overall_health: UNKNOWN } } }
I20260807 08:34:47.170189 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:47.165978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36232 (local address 127.9.174.3:46327)
0807 08:34:47.166149 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:47.166156 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:47.166173 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:34:47.166287 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:47.166289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:47.166357 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:47.166485 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:47.166774 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.167432 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.168325 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:47.168586 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:47.168703 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:47.169059 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:47.169678 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:47.169824 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:47.169869 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 08:34:47.169936 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 08:34:47.175074 16167 tablet.cc:2369] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:34:47.177910 16096 raft_consensus.cc:1261] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [term 1 FOLLOWER]: Refusing update from remote peer e4dcc489eaa1468fa11011e1f700a8dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:47.177940 15903 raft_consensus.cc:1261] T f50b4b97152d4509b129f4144adde7fd P e26a88c37cbc4082b86158d99a37d06f [term 1 FOLLOWER]: Refusing update from remote peer e4dcc489eaa1468fa11011e1f700a8dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:47.178634 16400 consensus_queue.cc:1035] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e19ea094b544f08a9a37eac7ef7d353" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 40931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:47.178848 16394 consensus_queue.cc:1035] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e26a88c37cbc4082b86158d99a37d06f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:47.188164 16403 mvcc.cc:205] Tried to move back new op lower bound from 7315831550671900672 to 7315831550509809664. Current Snapshot: MvccSnapshot[applied={T|T < 7315831550671900672}]
I20260807 08:34:47.190196 16405 mvcc.cc:205] Tried to move back new op lower bound from 7315831550671900672 to 7315831550509809664. Current Snapshot: MvccSnapshot[applied={T|T < 7315831550671900672}]
I20260807 08:34:47.200011 15705 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:47.200300 15705 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:47.203672 15705 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42c5e2faf742400ab4d8a09a9e87ccbf: Sending DeleteTablet for 3 replicas of tablet f50b4b97152d4509b129f4144adde7fd
I20260807 08:34:47.205034 16259 tablet_service.cc:1514] Processing DeleteTablet for tablet f50b4b97152d4509b129f4144adde7fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:47 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:36218
I20260807 08:34:47.205267 15879 tablet_service.cc:1514] Processing DeleteTablet for tablet f50b4b97152d4509b129f4144adde7fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:47 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:45708
I20260807 08:34:47.205281 16067 tablet_service.cc:1514] Processing DeleteTablet for tablet f50b4b97152d4509b129f4144adde7fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:47 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:34414
I20260807 08:34:47.205834 16414 tablet_replica.cc:330] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc: stopping tablet replica
I20260807 08:34:47.206144 16415 tablet_replica.cc:330] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353: stopping tablet replica
I20260807 08:34:47.206566 16415 raft_consensus.cc:2229] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:34:47.206787 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 15789
I20260807 08:34:47.207052 16415 raft_consensus.cc:2258] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:47.213687 16415 ts_tablet_manager.cc:1905] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:47.216408 16414 raft_consensus.cc:2229] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:34:47.217094 16414 raft_consensus.cc:2258] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:47.218261 16415 ts_tablet_manager.cc:1918] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:34:47.218410 16415 log.cc:1196] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/f50b4b97152d4509b129f4144adde7fd
I20260807 08:34:47.218823 16415 ts_tablet_manager.cc:1939] T f50b4b97152d4509b129f4144adde7fd P 4e19ea094b544f08a9a37eac7ef7d353: Deleting consensus metadata
I20260807 08:34:47.219554 16414 ts_tablet_manager.cc:1905] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:47.220794 15681 catalog_manager.cc:4841] TS 4e19ea094b544f08a9a37eac7ef7d353 (127.9.174.2:40931): tablet f50b4b97152d4509b129f4144adde7fd (table test-table [id=4a1ef8d63252498c97a2f1c0caf8828b]) successfully deleted
I20260807 08:34:47.223093 16414 ts_tablet_manager.cc:1918] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:34:47.223268 16414 log.cc:1196] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/f50b4b97152d4509b129f4144adde7fd
I20260807 08:34:47.223735 16414 ts_tablet_manager.cc:1939] T f50b4b97152d4509b129f4144adde7fd P e4dcc489eaa1468fa11011e1f700a8dc: Deleting consensus metadata
I20260807 08:34:47.225405 15683 catalog_manager.cc:4841] TS e4dcc489eaa1468fa11011e1f700a8dc (127.9.174.3:46327): tablet f50b4b97152d4509b129f4144adde7fd (table test-table [id=4a1ef8d63252498c97a2f1c0caf8828b]) successfully deleted
W20260807 08:34:47.226917 15686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:42981 (error 108)
I20260807 08:34:47.228186 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 15980
I20260807 08:34:47.228451 16374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:47.227997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:42981 (local address 127.0.0.1:45710)
0807 08:34:47.228189 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:47.228297 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:42981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 08:34:47.241593 15686 catalog_manager.cc:4584] TS e26a88c37cbc4082b86158d99a37d06f (127.9.174.1:42981): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f50b4b97152d4509b129f4144adde7fd: Network error: Client connection negotiation failed: client connection to 127.9.174.1:42981: connect: Connection refused (error 111)
I20260807 08:34:47.249881 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 16176
I20260807 08:34:47.266002 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 15658
2026-08-07T08:34:47Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2961 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T08:34:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:47Z Disabled control of system clock
I20260807 08:34:47.308210 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:33957
--webserver_interface=127.9.174.62
--webserver_port=0
--builtin_ntp_servers=127.9.174.20:40941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:33957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 08:34:47.443476 16425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:47.443900 16425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:47.444000 16425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:47.448500 16425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:47.448606 16425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:47.448642 16425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:47.448683 16425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:47.448719 16425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:47.453779 16425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:40941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:33957
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:33957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:47.455209 16425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:47.456868 16425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:47.467051 16431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:47.467109 16435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:47.467396 16425 server_base.cc:1034] running on GCE node
W20260807 08:34:47.467047 16430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:47.468340 16425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:47.469853 16425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:47.471093 16425 hybrid_clock.cc:648] HybridClock initialized: now 1786091687471074 us; error 37 us; skew 500 ppm
I20260807 08:34:47.473690 16425 webserver.cc:460] Webserver started at http://127.9.174.62:33377/ using document root <none> and password file <none>
I20260807 08:34:47.474478 16425 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:47.474555 16425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:47.474838 16425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:47.477631 16425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "1ee0a32159cf4bc288bcf2e7a79677df"
format_stamp: "Formatted at 2026-08-07 08:34:47 on dist-test-slave-h3ft"
server_key: "1ecb763f4fc426613c449b70ab35c6e6"
server_key_iv: "95026c0f39161f3d7cc69fda92e4be7f"
server_key_version: "encryptionkey@0"
I20260807 08:34:47.478375 16425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "1ee0a32159cf4bc288bcf2e7a79677df"
format_stamp: "Formatted at 2026-08-07 08:34:47 on dist-test-slave-h3ft"
server_key: "1ecb763f4fc426613c449b70ab35c6e6"
server_key_iv: "95026c0f39161f3d7cc69fda92e4be7f"
server_key_version: "encryptionkey@0"
I20260807 08:34:47.483405 16425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 08:34:47.486706 16441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:47.488610 16425 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:34:47.488785 16425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "1ee0a32159cf4bc288bcf2e7a79677df"
format_stamp: "Formatted at 2026-08-07 08:34:47 on dist-test-slave-h3ft"
server_key: "1ecb763f4fc426613c449b70ab35c6e6"
server_key_iv: "95026c0f39161f3d7cc69fda92e4be7f"
server_key_version: "encryptionkey@0"
I20260807 08:34:47.488930 16425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:47.494931 16425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:47.495862 16425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:47.496098 16425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:47.505790 16425 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:33957
I20260807 08:34:47.505872 16532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:33957 every 8 connection(s)
I20260807 08:34:47.507375 16425 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:47.512368 16533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:47.516130 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 16425
I20260807 08:34:47.516291 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:34:47.516655 9912 external_mini_cluster.cc:1428] Setting key 34e15c1565ee0c4b166eb15a811feccc
I20260807 08:34:47.519753 16533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df: Bootstrap starting.
I20260807 08:34:47.522755 16533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:47.523753 16533 log.cc:824] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:47.524139 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:47.518649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57108 (local address 127.9.174.62:33957)
0807 08:34:47.519547 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:47.519568 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:47.519617 (+ 49us) server_negotiation.cc:407] Connection header received
0807 08:34:47.520540 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:47.520562 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:47.521008 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:47.521376 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:47.522401 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:47.522407 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:47.522420 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:34:47.522431 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:47.522502 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:34:47.522522 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:47.522625 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:47.522812 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 08:34:47.523184 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":171,"threads_started":1}
I20260807 08:34:47.526572 16533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df: No bootstrap required, opened a new log
I20260807 08:34:47.530537 16533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee0a32159cf4bc288bcf2e7a79677df" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33957 } }
I20260807 08:34:47.530795 16533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:47.530833 16533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee0a32159cf4bc288bcf2e7a79677df, State: Initialized, Role: FOLLOWER
I20260807 08:34:47.531360 16533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee0a32159cf4bc288bcf2e7a79677df" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33957 } }
I20260807 08:34:47.531502 16533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:34:47.531548 16533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:34:47.531635 16533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:47.532771 16533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee0a32159cf4bc288bcf2e7a79677df" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33957 } }
I20260807 08:34:47.533182 16533 leader_election.cc:302] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ee0a32159cf4bc288bcf2e7a79677df; no voters:
I20260807 08:34:47.533636 16533 leader_election.cc:288] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:34:47.533812 16539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:47.534188 16539 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [term 1 LEADER]: Becoming Leader. State: Replica: 1ee0a32159cf4bc288bcf2e7a79677df, State: Running, Role: LEADER
I20260807 08:34:47.534614 16539 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee0a32159cf4bc288bcf2e7a79677df" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33957 } }
I20260807 08:34:47.535168 16533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:47.537010 16540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ee0a32159cf4bc288bcf2e7a79677df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee0a32159cf4bc288bcf2e7a79677df" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33957 } } }
I20260807 08:34:47.537070 16541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ee0a32159cf4bc288bcf2e7a79677df. Latest consensus state: current_term: 1 leader_uuid: "1ee0a32159cf4bc288bcf2e7a79677df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee0a32159cf4bc288bcf2e7a79677df" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33957 } } }
I20260807 08:34:47.537199 16540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:47.537204 16541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:47.537652 16557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:47.541534 16557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:47.548570 16557 catalog_manager.cc:1347] Generated new cluster ID: a001d6292e1b4b53bc9d17ec70550f50
I20260807 08:34:47.548688 16557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:47.557792 16557 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:47.558305 16557 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:47.572423 16557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df: Generated new TSK 0
I20260807 08:34:47.573433 16557 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:34:47.586609 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:33957
--builtin_ntp_servers=127.9.174.20:40941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 08:34:47.760536 16562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:47.760950 16562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:47.761034 16562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:47.765491 16562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:47.765630 16562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:47.765743 16562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:34:47.771227 16562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:40941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:33957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:47.772778 16562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:47.774617 16562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:47.787799 16571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:47.787786 16568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:47.787786 16569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:47.788388 16562 server_base.cc:1034] running on GCE node
I20260807 08:34:47.788969 16562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:47.789793 16562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:47.791047 16562 hybrid_clock.cc:648] HybridClock initialized: now 1786091687790968 us; error 71 us; skew 500 ppm
I20260807 08:34:47.793377 16562 webserver.cc:460] Webserver started at http://127.9.174.1:37759/ using document root <none> and password file <none>
I20260807 08:34:47.794242 16562 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:47.794348 16562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:47.794616 16562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:47.797123 16562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "024ad57bb1a14d02943c7cfa3f5a16e4"
format_stamp: "Formatted at 2026-08-07 08:34:47 on dist-test-slave-h3ft"
server_key: "894d9394c2f68616ac95b26b82f6de86"
server_key_iv: "d89db98a13081200bb45465d77b781b2"
server_key_version: "encryptionkey@0"
I20260807 08:34:47.797773 16562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "024ad57bb1a14d02943c7cfa3f5a16e4"
format_stamp: "Formatted at 2026-08-07 08:34:47 on dist-test-slave-h3ft"
server_key: "894d9394c2f68616ac95b26b82f6de86"
server_key_iv: "d89db98a13081200bb45465d77b781b2"
server_key_version: "encryptionkey@0"
I20260807 08:34:47.802348 16562 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 08:34:47.805531 16584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:47.807268 16562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:34:47.807386 16562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "024ad57bb1a14d02943c7cfa3f5a16e4"
format_stamp: "Formatted at 2026-08-07 08:34:47 on dist-test-slave-h3ft"
server_key: "894d9394c2f68616ac95b26b82f6de86"
server_key_iv: "d89db98a13081200bb45465d77b781b2"
server_key_version: "encryptionkey@0"
I20260807 08:34:47.807492 16562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:47.812543 16562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:47.813210 16562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:47.813369 16562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:47.813932 16562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:47.814915 16562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:47.814971 16562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:47.815068 16562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:47.815102 16562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:47.825910 16562 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:39525
I20260807 08:34:47.825989 16747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:39525 every 8 connection(s)
I20260807 08:34:47.826911 16562 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:34:47.828120 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 16562
I20260807 08:34:47.828287 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:34:47.828593 9912 external_mini_cluster.cc:1428] Setting key a367b9bee8dcac3c86bf9841a8dcf4ac
I20260807 08:34:47.831387 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:33957
--builtin_ntp_servers=127.9.174.20:40941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 08:34:47.835323 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:47.829483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:39099 (local address 127.9.174.62:33957)
0807 08:34:47.829663 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:47.829668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:47.831134 (+ 1466us) server_negotiation.cc:407] Connection header received
0807 08:34:47.832790 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:47.832794 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:47.832843 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:47.832919 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:47.834564 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:47.834568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:47.834571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:34:47.834620 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:47.834654 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:34:47.834666 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:47.834850 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:47.835137 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 08:34:47.835202 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 08:34:47.836706 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:47.829920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:33957 (local address 127.9.174.1:39099)
0807 08:34:47.830927 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:47.830983 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:47.832533 (+ 1550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:47.833164 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:47.833177 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:47.833325 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:47.834141 (+ 816us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:34:47.834144 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:47.834213 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:34:47.834326 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:47.834832 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:47.834846 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:47.834853 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 08:34:47.835122 (+ 269us) client_negotiation.cc:240] Negotiation successful
0807 08:34:47.835426 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":137,"threads_started":1}
I20260807 08:34:47.838325 16750 heartbeater.cc:347] Connected to a master server at 127.9.174.62:33957
I20260807 08:34:47.838666 16750 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:47.839380 16750 heartbeater.cc:511] Master 127.9.174.62:33957 requested a full tablet report, sending...
I20260807 08:34:47.841773 16469 ts_manager.cc:194] Registered new tserver with Master: 024ad57bb1a14d02943c7cfa3f5a16e4 (127.9.174.1:39525)
W20260807 08:34:47.971305 16754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:47.971683 16754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:47.971769 16754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:47.976157 16754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:47.976265 16754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:47.976372 16754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:34:47.981817 16754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:40941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:33957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:47.983399 16754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:47.985177 16754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:47.998257 16763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:47.998404 16761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:47.998333 16765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:47.998672 16754 server_base.cc:1034] running on GCE node
I20260807 08:34:47.999441 16754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:48.000262 16754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:48.001439 16754 hybrid_clock.cc:648] HybridClock initialized: now 1786091688001396 us; error 55 us; skew 500 ppm
I20260807 08:34:48.003958 16754 webserver.cc:460] Webserver started at http://127.9.174.2:40441/ using document root <none> and password file <none>
I20260807 08:34:48.004732 16754 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:48.004808 16754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:48.005086 16754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:48.007896 16754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "7023f40d3812413f92cdd16baf248d03"
format_stamp: "Formatted at 2026-08-07 08:34:48 on dist-test-slave-h3ft"
server_key: "aea13fa36987674996889d0e480f92ce"
server_key_iv: "957a0a678ebf0bf523004834ec1a2e0e"
server_key_version: "encryptionkey@0"
I20260807 08:34:48.008575 16754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "7023f40d3812413f92cdd16baf248d03"
format_stamp: "Formatted at 2026-08-07 08:34:48 on dist-test-slave-h3ft"
server_key: "aea13fa36987674996889d0e480f92ce"
server_key_iv: "957a0a678ebf0bf523004834ec1a2e0e"
server_key_version: "encryptionkey@0"
I20260807 08:34:48.013603 16754 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:34:48.016855 16775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:48.018541 16754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 08:34:48.018707 16754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "7023f40d3812413f92cdd16baf248d03"
format_stamp: "Formatted at 2026-08-07 08:34:48 on dist-test-slave-h3ft"
server_key: "aea13fa36987674996889d0e480f92ce"
server_key_iv: "957a0a678ebf0bf523004834ec1a2e0e"
server_key_version: "encryptionkey@0"
I20260807 08:34:48.018849 16754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:48.024394 16754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:48.025169 16754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:48.025382 16754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:48.026093 16754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:48.027401 16754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:48.027479 16754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:48.027560 16754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:48.027595 16754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:48.040191 16754 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:36987
I20260807 08:34:48.040308 16938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:36987 every 8 connection(s)
I20260807 08:34:48.041471 16754 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:34:48.042196 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 16754
I20260807 08:34:48.042378 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:34:48.042810 9912 external_mini_cluster.cc:1428] Setting key 848b158943ad4d63bca2b7246225b8e4
I20260807 08:34:48.045858 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:33957
--builtin_ntp_servers=127.9.174.20:40941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 08:34:48.051193 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.044378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:52723 (local address 127.9.174.62:33957)
0807 08:34:48.044527 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:48.044530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:48.046549 (+ 2019us) server_negotiation.cc:407] Connection header received
0807 08:34:48.048852 (+ 2303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:48.048857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:48.048918 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:48.049039 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:48.050523 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:48.050530 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:48.050536 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:34:48.050555 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:48.050587 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:34:48.050601 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:48.050714 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:48.050883 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 08:34:48.050943 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:34:48.052423 16944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.045145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:33957 (local address 127.9.174.2:52723)
0807 08:34:48.046316 (+ 1171us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:48.046392 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:48.048576 (+ 2184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:48.049295 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:48.049303 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:48.049408 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:48.050185 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:34:48.050187 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:48.050233 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:34:48.050315 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:48.050713 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:48.050722 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:48.050728 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:34:48.050899 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 08:34:48.051238 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":149,"threads_started":1}
I20260807 08:34:48.053916 16939 heartbeater.cc:347] Connected to a master server at 127.9.174.62:33957
I20260807 08:34:48.054215 16939 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:48.054731 16939 heartbeater.cc:511] Master 127.9.174.62:33957 requested a full tablet report, sending...
I20260807 08:34:48.056085 16469 ts_manager.cc:194] Registered new tserver with Master: 7023f40d3812413f92cdd16baf248d03 (127.9.174.2:36987)
W20260807 08:34:48.185922 16945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:48.186290 16945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:48.186335 16945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:48.190856 16945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:48.190938 16945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:48.191063 16945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:34:48.196538 16945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:40941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:33957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:48.198010 16945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:48.199864 16945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:48.211699 16952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:48.211812 16954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:48.211727 16951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:48.212146 16945 server_base.cc:1034] running on GCE node
I20260807 08:34:48.212781 16945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:48.213591 16945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:48.214798 16945 hybrid_clock.cc:648] HybridClock initialized: now 1786091688214752 us; error 62 us; skew 500 ppm
I20260807 08:34:48.226421 16945 webserver.cc:460] Webserver started at http://127.9.174.3:36969/ using document root <none> and password file <none>
I20260807 08:34:48.227329 16945 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:48.227443 16945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:48.227738 16945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:48.230455 16945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "ddc4f69a12e048abb562c1fb35572b76"
format_stamp: "Formatted at 2026-08-07 08:34:48 on dist-test-slave-h3ft"
server_key: "4ae146d7a359bd4d48ac9dea334a3ba5"
server_key_iv: "8a9cfaf1da301b4ee9f39f7d30a5bc5b"
server_key_version: "encryptionkey@0"
I20260807 08:34:48.231216 16945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "ddc4f69a12e048abb562c1fb35572b76"
format_stamp: "Formatted at 2026-08-07 08:34:48 on dist-test-slave-h3ft"
server_key: "4ae146d7a359bd4d48ac9dea334a3ba5"
server_key_iv: "8a9cfaf1da301b4ee9f39f7d30a5bc5b"
server_key_version: "encryptionkey@0"
I20260807 08:34:48.236608 16945 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 08:34:48.240087 16963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:48.241751 16945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:34:48.241925 16945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "ddc4f69a12e048abb562c1fb35572b76"
format_stamp: "Formatted at 2026-08-07 08:34:48 on dist-test-slave-h3ft"
server_key: "4ae146d7a359bd4d48ac9dea334a3ba5"
server_key_iv: "8a9cfaf1da301b4ee9f39f7d30a5bc5b"
server_key_version: "encryptionkey@0"
I20260807 08:34:48.242079 16945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:48.247761 16945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:48.248533 16945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:48.248749 16945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:48.249521 16945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:48.250691 16945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:48.250748 16945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:48.250792 16945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:48.250808 16945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:48.263348 16945 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:40271
I20260807 08:34:48.263532 17123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:40271 every 8 connection(s)
I20260807 08:34:48.264581 16945 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:34:48.266252 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 16945
I20260807 08:34:48.266419 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:34:48.266801 9912 external_mini_cluster.cc:1428] Setting key 60cb6cfd897397676286b7c01960118f
I20260807 08:34:48.273705 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.267482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:36223 (local address 127.9.174.62:33957)
0807 08:34:48.267692 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:48.267698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:48.269938 (+ 2240us) server_negotiation.cc:407] Connection header received
0807 08:34:48.271567 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:48.271570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:48.271621 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:48.271718 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:48.273048 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:48.273052 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:48.273054 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:34:48.273067 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:48.273092 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:34:48.273101 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:48.273187 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:48.273479 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 08:34:48.273541 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 08:34:48.274718 17126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.268067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:33957 (local address 127.9.174.3:36223)
0807 08:34:48.269768 (+ 1701us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:48.269824 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:48.271337 (+ 1513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:48.271896 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:48.271903 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:48.272015 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:48.272740 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:34:48.272742 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:48.272790 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:34:48.272873 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:48.273235 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:48.273250 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:48.273257 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 08:34:48.273473 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 08:34:48.273739 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":132,"threads_started":1}
I20260807 08:34:48.276237 17124 heartbeater.cc:347] Connected to a master server at 127.9.174.62:33957
I20260807 08:34:48.276530 17124 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:48.277078 17124 heartbeater.cc:511] Master 127.9.174.62:33957 requested a full tablet report, sending...
I20260807 08:34:48.278563 16469 ts_manager.cc:194] Registered new tserver with Master: ddc4f69a12e048abb562c1fb35572b76 (127.9.174.3:40271)
I20260807 08:34:48.281630 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:34:48.285951 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.283982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57110 (local address 127.9.174.62:33957)
0807 08:34:48.284180 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:48.284187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:48.284354 (+ 167us) server_negotiation.cc:407] Connection header received
0807 08:34:48.284556 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:48.284559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:48.284602 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:48.284689 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:48.285493 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:48.285497 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:48.285500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:34:48.285520 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:48.285542 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:34:48.285550 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:48.285649 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:48.285717 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 08:34:48.285771 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 08:34:48.289433 16469 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:34:48.293358 16469 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:34:48.309561 17140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.305479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:40271 (local address 127.0.0.1:51860)
0807 08:34:48.306618 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:48.306634 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:48.306729 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:48.307695 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:48.307702 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:48.307727 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:48.308166 (+ 439us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:34:48.308168 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:48.308190 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:34:48.308267 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:48.309273 (+ 1006us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:48.309276 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:48.309279 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:34:48.309374 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 08:34:48.309402 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1068,"thread_start_us":125,"threads_started":1}
I20260807 08:34:48.309561 17137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.305273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:39525 (local address 127.0.0.1:42538)
0807 08:34:48.305854 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:48.305917 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:48.306068 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:48.306968 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:48.306978 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:48.307105 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:48.308602 (+ 1497us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:34:48.308606 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:48.308634 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:34:48.308678 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:48.309103 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:48.309116 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:48.309123 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 08:34:48.309260 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 08:34:48.309350 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":128,"threads_started":1}
I20260807 08:34:48.309751 17138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.305479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42538 (local address 127.9.174.1:39525)
0807 08:34:48.305926 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:48.305936 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:48.306105 (+ 169us) server_negotiation.cc:407] Connection header received
0807 08:34:48.306210 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:48.306219 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:48.306461 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:48.306707 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:48.308849 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:48.308858 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:48.308875 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:34:48.308891 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:48.308956 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:34:48.308992 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:48.309136 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:48.309396 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 08:34:48.309502 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":90,"threads_started":1}
I20260807 08:34:48.309945 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.306229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:36987 (local address 127.0.0.1:50980)
0807 08:34:48.306833 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:48.306853 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:48.306948 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:48.308396 (+ 1448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:48.308400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:48.308420 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:48.309066 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:34:48.309069 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:48.309080 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:34:48.309106 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:48.309659 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:48.309662 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:48.309666 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:34:48.309742 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 08:34:48.309791 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":146,"threads_started":1}
I20260807 08:34:48.310487 17143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.306519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50980 (local address 127.9.174.2:36987)
0807 08:34:48.307170 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:48.307180 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:48.307205 (+ 25us) server_negotiation.cc:407] Connection header received
0807 08:34:48.307261 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:48.307266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:48.307814 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:48.308013 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:48.309311 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:48.309321 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:48.309335 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:34:48.309356 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:48.309411 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:34:48.309465 (+ 54us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:48.309596 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:48.310241 (+ 645us) server_negotiation.cc:299] Negotiation successful
0807 08:34:48.310321 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":333,"threads_started":1}
I20260807 08:34:48.311671 17141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.306316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51860 (local address 127.9.174.3:40271)
0807 08:34:48.307227 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:48.307243 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:48.307270 (+ 27us) server_negotiation.cc:407] Connection header received
0807 08:34:48.307329 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:48.307336 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:48.307514 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:48.307954 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:48.309043 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:48.309050 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:48.309062 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:34:48.309072 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:48.309126 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:34:48.309160 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:48.309282 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:48.310484 (+ 1202us) server_negotiation.cc:299] Negotiation successful
0807 08:34:48.311456 (+ 972us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":799,"thread_start_us":186,"threads_started":1}
I20260807 08:34:48.313313 16651 tablet_service.cc:1467] Processing CreateTablet for tablet 063d4eb0c39147e88b2512fdfd1333c8 (DEFAULT_TABLE table=test-table [id=b476bad21c9e4e20811ced7202cc8c1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:48.313932 16844 tablet_service.cc:1467] Processing CreateTablet for tablet 063d4eb0c39147e88b2512fdfd1333c8 (DEFAULT_TABLE table=test-table [id=b476bad21c9e4e20811ced7202cc8c1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:48.314635 16651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 063d4eb0c39147e88b2512fdfd1333c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:48.314752 17032 tablet_service.cc:1467] Processing CreateTablet for tablet 063d4eb0c39147e88b2512fdfd1333c8 (DEFAULT_TABLE table=test-table [id=b476bad21c9e4e20811ced7202cc8c1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:48.315259 16844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 063d4eb0c39147e88b2512fdfd1333c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:48.315922 17032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 063d4eb0c39147e88b2512fdfd1333c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:48.323101 17145 tablet_bootstrap.cc:492] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03: Bootstrap starting.
I20260807 08:34:48.324550 17147 tablet_bootstrap.cc:492] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76: Bootstrap starting.
I20260807 08:34:48.326011 17145 tablet_bootstrap.cc:654] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:48.326951 17147 tablet_bootstrap.cc:654] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:48.326982 17145 log.cc:824] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:48.327320 17146 tablet_bootstrap.cc:492] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4: Bootstrap starting.
I20260807 08:34:48.327919 17147 log.cc:824] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:48.329897 17146 tablet_bootstrap.cc:654] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:48.330253 17145 tablet_bootstrap.cc:492] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03: No bootstrap required, opened a new log
I20260807 08:34:48.330499 17145 ts_tablet_manager.cc:1397] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s
I20260807 08:34:48.330827 17146 log.cc:824] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:48.333148 17146 tablet_bootstrap.cc:492] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4: No bootstrap required, opened a new log
I20260807 08:34:48.333406 17146 ts_tablet_manager.cc:1397] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260807 08:34:48.335067 17147 tablet_bootstrap.cc:492] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76: No bootstrap required, opened a new log
I20260807 08:34:48.334858 17145 raft_consensus.cc:348] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024ad57bb1a14d02943c7cfa3f5a16e4" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 39525 } } peers { permanent_uuid: "ddc4f69a12e048abb562c1fb35572b76" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40271 } } peers { permanent_uuid: "7023f40d3812413f92cdd16baf248d03" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36987 } }
I20260807 08:34:48.335314 17145 raft_consensus.cc:374] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:48.335352 17147 ts_tablet_manager.cc:1397] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260807 08:34:48.335383 17145 raft_consensus.cc:729] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7023f40d3812413f92cdd16baf248d03, State: Initialized, Role: FOLLOWER
I20260807 08:34:48.335920 17145 consensus_queue.cc:260] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024ad57bb1a14d02943c7cfa3f5a16e4" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 39525 } } peers { permanent_uuid: "ddc4f69a12e048abb562c1fb35572b76" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40271 } } peers { permanent_uuid: "7023f40d3812413f92cdd16baf248d03" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36987 } }
I20260807 08:34:48.336890 16939 heartbeater.cc:503] Master 127.9.174.62:33957 was elected leader, sending a full tablet report...
I20260807 08:34:48.337282 17145 ts_tablet_manager.cc:1428] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260807 08:34:48.339133 17146 raft_consensus.cc:348] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024ad57bb1a14d02943c7cfa3f5a16e4" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 39525 } } peers { permanent_uuid: "ddc4f69a12e048abb562c1fb35572b76" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40271 } } peers { permanent_uuid: "7023f40d3812413f92cdd16baf248d03" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36987 } }
I20260807 08:34:48.339497 17146 raft_consensus.cc:374] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:48.339563 17146 raft_consensus.cc:729] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 024ad57bb1a14d02943c7cfa3f5a16e4, State: Initialized, Role: FOLLOWER
I20260807 08:34:48.340150 17146 consensus_queue.cc:260] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024ad57bb1a14d02943c7cfa3f5a16e4" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 39525 } } peers { permanent_uuid: "ddc4f69a12e048abb562c1fb35572b76" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40271 } } peers { permanent_uuid: "7023f40d3812413f92cdd16baf248d03" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36987 } }
I20260807 08:34:48.340626 17147 raft_consensus.cc:348] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024ad57bb1a14d02943c7cfa3f5a16e4" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 39525 } } peers { permanent_uuid: "ddc4f69a12e048abb562c1fb35572b76" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40271 } } peers { permanent_uuid: "7023f40d3812413f92cdd16baf248d03" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36987 } }
I20260807 08:34:48.340926 17147 raft_consensus.cc:374] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:48.340994 17147 raft_consensus.cc:729] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddc4f69a12e048abb562c1fb35572b76, State: Initialized, Role: FOLLOWER
I20260807 08:34:48.341017 16750 heartbeater.cc:503] Master 127.9.174.62:33957 was elected leader, sending a full tablet report...
I20260807 08:34:48.341537 17147 consensus_queue.cc:260] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024ad57bb1a14d02943c7cfa3f5a16e4" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 39525 } } peers { permanent_uuid: "ddc4f69a12e048abb562c1fb35572b76" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40271 } } peers { permanent_uuid: "7023f40d3812413f92cdd16baf248d03" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36987 } }
I20260807 08:34:48.342749 17146 ts_tablet_manager.cc:1428] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260807 08:34:48.343659 17147 ts_tablet_manager.cc:1428] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76: Time spent starting tablet: real 0.008s user 0.002s sys 0.004s
I20260807 08:34:48.344071 17124 heartbeater.cc:503] Master 127.9.174.62:33957 was elected leader, sending a full tablet report...
I20260807 08:34:48.481333 17157 raft_consensus.cc:482] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:34:48.481628 17157 raft_consensus.cc:504] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024ad57bb1a14d02943c7cfa3f5a16e4" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 39525 } } peers { permanent_uuid: "ddc4f69a12e048abb562c1fb35572b76" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40271 } } peers { permanent_uuid: "7023f40d3812413f92cdd16baf248d03" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36987 } }
I20260807 08:34:48.483206 17157 leader_election.cc:288] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 024ad57bb1a14d02943c7cfa3f5a16e4 (127.9.174.1:39525), 7023f40d3812413f92cdd16baf248d03 (127.9.174.2:36987)
I20260807 08:34:48.485845 17126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.483704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:39525 (local address 127.9.174.3:49615)
0807 08:34:48.483893 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:48.483912 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:48.484031 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:48.484414 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:48.484419 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:48.484443 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:48.485134 (+ 691us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:34:48.485135 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:48.485147 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:34:48.485176 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:48.485512 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:48.485516 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:48.485520 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:34:48.485626 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 08:34:48.485673 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 08:34:48.485853 17138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.483904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:49615 (local address 127.9.174.1:39525)
0807 08:34:48.484041 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:48.484044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:48.484062 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:34:48.484124 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:48.484126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:48.484164 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:48.484346 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:48.485366 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:48.485369 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:48.485371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:34:48.485386 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:48.485413 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:34:48.485423 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:48.485541 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:48.485620 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 08:34:48.485674 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:34:48.486461 17143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.483783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:39853 (local address 127.9.174.2:36987)
0807 08:34:48.483972 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:48.483978 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:48.484649 (+ 671us) server_negotiation.cc:407] Connection header received
0807 08:34:48.484756 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:48.484759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:48.484808 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:48.484930 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:48.485761 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:48.485765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:48.485768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:34:48.485789 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:48.485815 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:34:48.485824 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:48.485931 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:48.486234 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 08:34:48.486295 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 08:34:48.486475 17158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.483704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:36987 (local address 127.9.174.3:39853)
0807 08:34:48.484507 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:48.484523 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:48.484629 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:48.484988 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:48.484993 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:48.485011 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:48.485556 (+ 545us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:34:48.485558 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:48.485570 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:34:48.485601 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:48.486075 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:48.486082 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:48.486085 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:34:48.486210 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 08:34:48.486254 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":264,"thread_start_us":197,"threads_started":1}
I20260807 08:34:48.486773 16682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "063d4eb0c39147e88b2512fdfd1333c8" candidate_uuid: "ddc4f69a12e048abb562c1fb35572b76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "024ad57bb1a14d02943c7cfa3f5a16e4" is_pre_election: true
I20260807 08:34:48.487183 16682 raft_consensus.cc:2454] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddc4f69a12e048abb562c1fb35572b76 in term 0.
I20260807 08:34:48.487493 16871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "063d4eb0c39147e88b2512fdfd1333c8" candidate_uuid: "ddc4f69a12e048abb562c1fb35572b76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7023f40d3812413f92cdd16baf248d03" is_pre_election: true
I20260807 08:34:48.487875 16964 leader_election.cc:302] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 024ad57bb1a14d02943c7cfa3f5a16e4, ddc4f69a12e048abb562c1fb35572b76; no voters:
I20260807 08:34:48.487931 16871 raft_consensus.cc:2454] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddc4f69a12e048abb562c1fb35572b76 in term 0.
I20260807 08:34:48.488315 17157 raft_consensus.cc:2781] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:34:48.488444 17157 raft_consensus.cc:482] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:34:48.488518 17157 raft_consensus.cc:3037] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:48.489691 17157 raft_consensus.cc:504] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024ad57bb1a14d02943c7cfa3f5a16e4" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 39525 } } peers { permanent_uuid: "ddc4f69a12e048abb562c1fb35572b76" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40271 } } peers { permanent_uuid: "7023f40d3812413f92cdd16baf248d03" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36987 } }
I20260807 08:34:48.490113 17157 leader_election.cc:288] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [CANDIDATE]: Term 1 election: Requested vote from peers 024ad57bb1a14d02943c7cfa3f5a16e4 (127.9.174.1:39525), 7023f40d3812413f92cdd16baf248d03 (127.9.174.2:36987)
I20260807 08:34:48.490633 16682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "063d4eb0c39147e88b2512fdfd1333c8" candidate_uuid: "ddc4f69a12e048abb562c1fb35572b76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "024ad57bb1a14d02943c7cfa3f5a16e4"
I20260807 08:34:48.490830 16682 raft_consensus.cc:3037] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:48.492388 16682 raft_consensus.cc:2454] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddc4f69a12e048abb562c1fb35572b76 in term 1.
I20260807 08:34:48.492803 16871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "063d4eb0c39147e88b2512fdfd1333c8" candidate_uuid: "ddc4f69a12e048abb562c1fb35572b76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7023f40d3812413f92cdd16baf248d03"
I20260807 08:34:48.492841 16964 leader_election.cc:302] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 024ad57bb1a14d02943c7cfa3f5a16e4, ddc4f69a12e048abb562c1fb35572b76; no voters:
I20260807 08:34:48.492985 16871 raft_consensus.cc:3037] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:48.493148 17157 raft_consensus.cc:2781] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:48.493466 17157 raft_consensus.cc:686] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [term 1 LEADER]: Becoming Leader. State: Replica: ddc4f69a12e048abb562c1fb35572b76, State: Running, Role: LEADER
I20260807 08:34:48.493957 17157 consensus_queue.cc:237] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024ad57bb1a14d02943c7cfa3f5a16e4" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 39525 } } peers { permanent_uuid: "ddc4f69a12e048abb562c1fb35572b76" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40271 } } peers { permanent_uuid: "7023f40d3812413f92cdd16baf248d03" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36987 } }
I20260807 08:34:48.494501 16871 raft_consensus.cc:2454] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddc4f69a12e048abb562c1fb35572b76 in term 1.
I20260807 08:34:48.498353 16466 catalog_manager.cc:5495] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 reported cstate change: term changed from 0 to 1, leader changed from <none> to ddc4f69a12e048abb562c1fb35572b76 (127.9.174.3). New cstate: current_term: 1 leader_uuid: "ddc4f69a12e048abb562c1fb35572b76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024ad57bb1a14d02943c7cfa3f5a16e4" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 39525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddc4f69a12e048abb562c1fb35572b76" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7023f40d3812413f92cdd16baf248d03" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36987 } health_report { overall_health: UNKNOWN } } }
W20260807 08:34:48.515810 17125 tablet.cc:2369] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:34:48.543084 16943 tablet.cc:2369] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:34:48.551761 17141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.549789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57084 (local address 127.9.174.3:40271)
0807 08:34:48.550006 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:48.550011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:48.550027 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:34:48.550135 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:48.550137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:48.550213 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:48.550302 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:48.551208 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:48.551212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:48.551214 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:34:48.551226 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:48.551261 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:34:48.551270 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:48.551362 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:48.551555 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 08:34:48.551623 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260807 08:34:48.559624 16682 raft_consensus.cc:1261] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4 [term 1 FOLLOWER]: Refusing update from remote peer ddc4f69a12e048abb562c1fb35572b76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:48.559624 16871 raft_consensus.cc:1261] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03 [term 1 FOLLOWER]: Refusing update from remote peer ddc4f69a12e048abb562c1fb35572b76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:48.560597 17160 consensus_queue.cc:1035] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "024ad57bb1a14d02943c7cfa3f5a16e4" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 39525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:48.560907 17157 consensus_queue.cc:1035] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7023f40d3812413f92cdd16baf248d03" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:48.569487 17163 mvcc.cc:205] Tried to move back new op lower bound from 7315831556330381312 to 7315831556075151360. Current Snapshot: MvccSnapshot[applied={T|T < 7315831556330381312}]
I20260807 08:34:48.571251 17164 mvcc.cc:205] Tried to move back new op lower bound from 7315831556330381312 to 7315831556075151360. Current Snapshot: MvccSnapshot[applied={T|T < 7315831556330381312}]
I20260807 08:34:48.573232 17165 mvcc.cc:205] Tried to move back new op lower bound from 7315831556330381312 to 7315831556075151360. Current Snapshot: MvccSnapshot[applied={T|T < 7315831556330381312 or (T in {7315831556330381312})}]
I20260807 08:34:48.582190 16466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:48.582479 16466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:48.585834 16466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ee0a32159cf4bc288bcf2e7a79677df: Sending DeleteTablet for 3 replicas of tablet 063d4eb0c39147e88b2512fdfd1333c8
I20260807 08:34:48.586787 16844 tablet_service.cc:1514] Processing DeleteTablet for tablet 063d4eb0c39147e88b2512fdfd1333c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:48 UTC) from {username='slave'} at 127.0.0.1:50980
I20260807 08:34:48.586766 17032 tablet_service.cc:1514] Processing DeleteTablet for tablet 063d4eb0c39147e88b2512fdfd1333c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:48 UTC) from {username='slave'} at 127.0.0.1:51860
I20260807 08:34:48.587029 16651 tablet_service.cc:1514] Processing DeleteTablet for tablet 063d4eb0c39147e88b2512fdfd1333c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:48 UTC) from {username='slave'} at 127.0.0.1:42538
I20260807 08:34:48.587520 17175 tablet_replica.cc:330] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03: stopping tablet replica
I20260807 08:34:48.587761 17177 tablet_replica.cc:330] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4: stopping tablet replica
I20260807 08:34:48.588124 17175 raft_consensus.cc:2229] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:34:48.588189 17177 raft_consensus.cc:2229] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:34:48.588421 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 16562
I20260807 08:34:48.588594 17177 raft_consensus.cc:2258] T 063d4eb0c39147e88b2512fdfd1333c8 P 024ad57bb1a14d02943c7cfa3f5a16e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:48.588801 17175 raft_consensus.cc:2258] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:48.589359 17176 tablet_replica.cc:330] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76: stopping tablet replica
I20260807 08:34:48.589746 17176 raft_consensus.cc:2229] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:34:48.590233 17176 raft_consensus.cc:2258] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:48.593060 17176 ts_tablet_manager.cc:1905] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:48.593808 17175 ts_tablet_manager.cc:1905] T 063d4eb0c39147e88b2512fdfd1333c8 P 7023f40d3812413f92cdd16baf248d03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:48.597267 17176 ts_tablet_manager.cc:1918] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:34:48.597496 17176 log.cc:1196] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/063d4eb0c39147e88b2512fdfd1333c8
I20260807 08:34:48.597968 17176 ts_tablet_manager.cc:1939] T 063d4eb0c39147e88b2512fdfd1333c8 P ddc4f69a12e048abb562c1fb35572b76: Deleting consensus metadata
I20260807 08:34:48.599596 16445 catalog_manager.cc:4841] TS ddc4f69a12e048abb562c1fb35572b76 (127.9.174.3:40271): tablet 063d4eb0c39147e88b2512fdfd1333c8 (table test-table [id=b476bad21c9e4e20811ced7202cc8c1a]) successfully deleted
W20260807 08:34:48.603665 16443 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:39525 (error 108)
I20260807 08:34:48.603991 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 16754
I20260807 08:34:48.612067 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.611601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:39525 (local address 127.0.0.1:42058)
0807 08:34:48.611853 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:48.611942 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:39525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260807 08:34:48.612392 16443 catalog_manager.cc:4584] TS 024ad57bb1a14d02943c7cfa3f5a16e4 (127.9.174.1:39525): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 063d4eb0c39147e88b2512fdfd1333c8: Network error: Client connection negotiation failed: client connection to 127.9.174.1:39525: connect: Connection refused (error 111)
I20260807 08:34:48.621807 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 16945
I20260807 08:34:48.625988 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.622038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:36987 (local address 127.0.0.1:35380)
0807 08:34:48.625759 (+ 3721us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:48.625841 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.2:36987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3580}
I20260807 08:34:48.636476 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 16425
2026-08-07T08:34:48Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1370 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 08:34:48.667788 9912 mini_oidc.cc:152] Starting JWKS server
I20260807 08:34:48.669626 9912 webserver.cc:460] Webserver started at https://127.0.0.1:33441/ using document root <none> and password file <none>
I20260807 08:34:48.669790 9912 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 08:34:48.670050 9912 webserver.cc:460] Webserver started at http://127.0.0.1:46617/ using document root <none> and password file <none>
2026-08-07T08:34:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:48Z Disabled control of system clock
I20260807 08:34:48.683786 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:41621
--webserver_interface=127.9.174.62
--webserver_port=0
--builtin_ntp_servers=127.9.174.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:41621
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33441/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/testchainca.cert with env {}
W20260807 08:34:48.812264 17189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:48.812613 17189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:48.812662 17189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:48.817325 17189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:48.817401 17189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:48.817420 17189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:48.817440 17189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:48.817458 17189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 08:34:48.817476 17189 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 08:34:48.817492 17189 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33441/jwks.json
I20260807 08:34:48.823663 17189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:41621
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:41621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33441/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:48.824862 17189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:48.825963 17189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:48.836052 17197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:48.836087 17195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:48.836154 17194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:48.836704 17189 server_base.cc:1034] running on GCE node
I20260807 08:34:48.837662 17189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:48.839327 17189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:48.840523 17189 hybrid_clock.cc:648] HybridClock initialized: now 1786091688840496 us; error 45 us; skew 500 ppm
I20260807 08:34:48.843473 17189 webserver.cc:460] Webserver started at http://127.9.174.62:43791/ using document root <none> and password file <none>
I20260807 08:34:48.844580 17189 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:48.844705 17189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:48.845063 17189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:48.848692 17189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "4e42731206314fa996ed4d87e5e49a27"
format_stamp: "Formatted at 2026-08-07 08:34:48 on dist-test-slave-h3ft"
server_key: "557c2372ccb388822e9947d9b58d2d21"
server_key_iv: "ebad9e30ba48d008bb5661b18c49534e"
server_key_version: "encryptionkey@0"
I20260807 08:34:48.849498 17189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "4e42731206314fa996ed4d87e5e49a27"
format_stamp: "Formatted at 2026-08-07 08:34:48 on dist-test-slave-h3ft"
server_key: "557c2372ccb388822e9947d9b58d2d21"
server_key_iv: "ebad9e30ba48d008bb5661b18c49534e"
server_key_version: "encryptionkey@0"
I20260807 08:34:48.855758 17189 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 08:34:48.859411 17203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:48.861271 17189 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:34:48.861502 17189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "4e42731206314fa996ed4d87e5e49a27"
format_stamp: "Formatted at 2026-08-07 08:34:48 on dist-test-slave-h3ft"
server_key: "557c2372ccb388822e9947d9b58d2d21"
server_key_iv: "ebad9e30ba48d008bb5661b18c49534e"
server_key_version: "encryptionkey@0"
I20260807 08:34:48.861673 17189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:48.880914 17189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:48.881891 17189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:48.882140 17189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:48.891508 17189 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:41621
I20260807 08:34:48.891541 17283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:41621 every 8 connection(s)
I20260807 08:34:48.892899 17189 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:48.893512 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 17189
I20260807 08:34:48.893687 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:34:48.894074 9912 external_mini_cluster.cc:1428] Setting key 7f560958e699a2a804b36df39fa7070b
I20260807 08:34:48.898011 17284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:48.905548 17284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27: Bootstrap starting.
I20260807 08:34:48.906042 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.895904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42818 (local address 127.9.174.62:41621)
0807 08:34:48.896747 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:48.896772 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:48.896824 (+ 52us) server_negotiation.cc:407] Connection header received
0807 08:34:48.897709 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:48.897732 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:48.898373 (+ 641us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:48.898785 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:48.899328 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:48.900550 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:48.901841 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:48.902420 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:48.902560 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:48.903715 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:48.903721 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:48.903738 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:34:48.904228 (+ 490us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:48.904302 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:34:48.904324 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:48.904431 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:48.904625 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 08:34:48.905014 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":155,"threads_started":1}
I20260807 08:34:48.908371 17284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:48.909478 17284 log.cc:824] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:48.910013 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:34:48.912305 17284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27: No bootstrap required, opened a new log
I20260807 08:34:48.915949 17284 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e42731206314fa996ed4d87e5e49a27" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 41621 } }
I20260807 08:34:48.916194 17284 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:48.916231 17284 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e42731206314fa996ed4d87e5e49a27, State: Initialized, Role: FOLLOWER
I20260807 08:34:48.916684 17284 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e42731206314fa996ed4d87e5e49a27" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 41621 } }
I20260807 08:34:48.916862 17284 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:34:48.916929 17284 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:34:48.917069 17284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:48.918170 17284 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e42731206314fa996ed4d87e5e49a27" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 41621 } }
I20260807 08:34:48.918556 17284 leader_election.cc:302] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e42731206314fa996ed4d87e5e49a27; no voters:
I20260807 08:34:48.918951 17284 leader_election.cc:288] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:34:48.919193 17290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:48.919618 17290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [term 1 LEADER]: Becoming Leader. State: Replica: 4e42731206314fa996ed4d87e5e49a27, State: Running, Role: LEADER
I20260807 08:34:48.920059 17290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e42731206314fa996ed4d87e5e49a27" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 41621 } }
I20260807 08:34:48.920399 17284 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:48.922358 17292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e42731206314fa996ed4d87e5e49a27. Latest consensus state: current_term: 1 leader_uuid: "4e42731206314fa996ed4d87e5e49a27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e42731206314fa996ed4d87e5e49a27" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 41621 } } }
I20260807 08:34:48.922358 17291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e42731206314fa996ed4d87e5e49a27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e42731206314fa996ed4d87e5e49a27" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 41621 } } }
I20260807 08:34:48.922547 17291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:48.922547 17292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27 [sys.catalog]: This master's current role is: LEADER
W20260807 08:34:48.925690 17309 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:34:48.925794 17309 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:34:48.925881 17310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:48.926637 17310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:48.933596 17310 catalog_manager.cc:1347] Generated new cluster ID: a3faaf1584164f479f3fc7996858b430
I20260807 08:34:48.933683 17310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:48.950021 17310 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:48.951714 17310 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:48.965353 17310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e42731206314fa996ed4d87e5e49a27: Generated new TSK 0
I20260807 08:34:48.966499 17310 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 08:34:49.003908 17321 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 08:34:49.004060 17321 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 08:34:49.004976 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:48.983209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42822 (local address 127.9.174.62:41621)
0807 08:34:48.983407 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:48.983412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:48.983672 (+ 260us) server_negotiation.cc:407] Connection header received
0807 08:34:48.983958 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:48.983962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:48.984040 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:48.984216 (+ 176us) server_negotiation.cc:226] Negotiated authn=JWT
0807 08:34:48.984577 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:48.985495 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:48.986634 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:48.987109 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:48.987231 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:48.987675 (+ 444us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 08:34:49.004358 (+ 16683us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 08:34:49.004747 (+ 389us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42822: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 08:34:49.005087 17288 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42822: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 08:34:49.006418 17320 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:48.983074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:41621 (local address 127.0.0.1:42822)
0807 08:34:48.983574 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:48.983587 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:48.983714 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:48.984283 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:48.984286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:48.984295 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 08:34:48.984435 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:48.984440 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:48.985687 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:48.985691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:48.986485 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:48.986494 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:48.987256 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:48.987259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:48.987344 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:48.987405 (+ 61us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 08:34:49.004697 (+ 17292us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 08:34:49.004964 (+ 267us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.174.62:41621: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":157,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.174.62:41621: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33441/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 08:34:49.007469 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 17189
2026-08-07T08:34:49Z chronyd exiting
I20260807 08:34:49.063911 9912 test_util.cc:175] -----------------------------------------------
I20260807 08:34:49.064034 9912 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091655732880-9912-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (398 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 08:34:49.065299 9912 mini_oidc.cc:152] Starting JWKS server
I20260807 08:34:49.066280 9912 webserver.cc:460] Webserver started at https://127.0.0.1:39357/ using document root <none> and password file <none>
I20260807 08:34:49.066382 9912 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 08:34:49.066630 9912 webserver.cc:460] Webserver started at http://127.0.0.1:44777/ using document root <none> and password file <none>
2026-08-07T08:34:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:49Z Disabled control of system clock
I20260807 08:34:49.080850 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:38007
--webserver_interface=127.9.174.62
--webserver_port=0
--builtin_ntp_servers=127.9.174.20:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:38007
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39357/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/testchainca.cert with env {}
W20260807 08:34:49.214986 17330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:49.215354 17330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:49.215410 17330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:49.219684 17330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:49.219762 17330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:49.219779 17330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:49.219795 17330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:49.219812 17330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 08:34:49.219830 17330 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 08:34:49.219842 17330 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39357/jwks.json
I20260807 08:34:49.225807 17330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:38007
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:38007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39357/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:49.227270 17330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:49.228536 17330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:49.239068 17343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:49.239033 17341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:49.239377 17330 server_base.cc:1034] running on GCE node
W20260807 08:34:49.239106 17340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:49.240298 17330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:49.241840 17330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:49.243131 17330 hybrid_clock.cc:648] HybridClock initialized: now 1786091689243070 us; error 60 us; skew 500 ppm
I20260807 08:34:49.245541 17330 webserver.cc:460] Webserver started at http://127.9.174.62:43007/ using document root <none> and password file <none>
I20260807 08:34:49.246264 17330 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:49.246366 17330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:49.246625 17330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:49.249003 17330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "426de55c42f0499380c89c834b13a238"
format_stamp: "Formatted at 2026-08-07 08:34:49 on dist-test-slave-h3ft"
server_key: "6b7f1a99bfe1769226d0b3e512d8a7c1"
server_key_iv: "36ef63d02c0d38bd7ee1956258d996ca"
server_key_version: "encryptionkey@0"
I20260807 08:34:49.249668 17330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "426de55c42f0499380c89c834b13a238"
format_stamp: "Formatted at 2026-08-07 08:34:49 on dist-test-slave-h3ft"
server_key: "6b7f1a99bfe1769226d0b3e512d8a7c1"
server_key_iv: "36ef63d02c0d38bd7ee1956258d996ca"
server_key_version: "encryptionkey@0"
I20260807 08:34:49.254112 17330 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 08:34:49.257272 17353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:49.258893 17330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:34:49.259099 17330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "426de55c42f0499380c89c834b13a238"
format_stamp: "Formatted at 2026-08-07 08:34:49 on dist-test-slave-h3ft"
server_key: "6b7f1a99bfe1769226d0b3e512d8a7c1"
server_key_iv: "36ef63d02c0d38bd7ee1956258d996ca"
server_key_version: "encryptionkey@0"
I20260807 08:34:49.259246 17330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:49.273312 17330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:49.274096 17330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:49.274318 17330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:49.282590 17330 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:38007
I20260807 08:34:49.282627 17424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:38007 every 8 connection(s)
I20260807 08:34:49.283859 17330 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:49.287952 17425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:49.290354 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 17330
I20260807 08:34:49.290518 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:34:49.290887 9912 external_mini_cluster.cc:1428] Setting key 415530b395cb5cb80cfa99cf38f28deb
I20260807 08:34:49.295655 17425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238: Bootstrap starting.
I20260807 08:34:49.299021 17425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:49.300266 17425 log.cc:824] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:49.302345 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:49.292567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51394 (local address 127.9.174.62:38007)
0807 08:34:49.293358 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:49.293377 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:49.293420 (+ 43us) server_negotiation.cc:407] Connection header received
0807 08:34:49.294399 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:49.294423 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:49.294908 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:49.295330 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:49.295923 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:49.296970 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:49.298341 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:49.298882 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:49.299110 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:49.300206 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:49.300211 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:49.300226 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:34:49.300776 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:49.300862 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:34:49.300893 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:49.301007 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:49.301188 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 08:34:49.301506 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":164,"threads_started":1}
I20260807 08:34:49.303151 17425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238: No bootstrap required, opened a new log
I20260807 08:34:49.305727 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:34:49.308019 17425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426de55c42f0499380c89c834b13a238" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38007 } }
I20260807 08:34:49.308466 17425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:49.308547 17425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 426de55c42f0499380c89c834b13a238, State: Initialized, Role: FOLLOWER
I20260807 08:34:49.309063 17425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426de55c42f0499380c89c834b13a238" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38007 } }
I20260807 08:34:49.309242 17425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:34:49.309345 17425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:34:49.309468 17425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:49.310889 17425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426de55c42f0499380c89c834b13a238" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38007 } }
I20260807 08:34:49.311409 17425 leader_election.cc:302] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 426de55c42f0499380c89c834b13a238; no voters:
I20260807 08:34:49.311887 17425 leader_election.cc:288] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:34:49.312256 17432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:49.312566 17432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [term 1 LEADER]: Becoming Leader. State: Replica: 426de55c42f0499380c89c834b13a238, State: Running, Role: LEADER
I20260807 08:34:49.313073 17432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426de55c42f0499380c89c834b13a238" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38007 } }
I20260807 08:34:49.313623 17425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:49.314942 17433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "426de55c42f0499380c89c834b13a238" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426de55c42f0499380c89c834b13a238" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38007 } } }
I20260807 08:34:49.315164 17433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:49.315634 17443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:49.315693 17434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 426de55c42f0499380c89c834b13a238. Latest consensus state: current_term: 1 leader_uuid: "426de55c42f0499380c89c834b13a238" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426de55c42f0499380c89c834b13a238" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38007 } } }
I20260807 08:34:49.315896 17434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:49.319249 17443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:49.325070 17443 catalog_manager.cc:1347] Generated new cluster ID: 8865fa25ba7e4eacb9c61f95f108aa28
I20260807 08:34:49.325167 17443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:49.351686 17443 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:49.352828 17443 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:49.359695 17443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 426de55c42f0499380c89c834b13a238: Generated new TSK 0
I20260807 08:34:49.360900 17443 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 08:34:49.394567 17464 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 08:34:49.394692 17464 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 08:34:49.395452 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:49.378595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51398 (local address 127.9.174.62:38007)
0807 08:34:49.378801 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:49.378808 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:49.379139 (+ 331us) server_negotiation.cc:407] Connection header received
0807 08:34:49.379341 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:49.379344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:49.379400 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:49.379513 (+ 113us) server_negotiation.cc:226] Negotiated authn=JWT
0807 08:34:49.379898 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:49.380535 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:49.381898 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:49.382138 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:49.382240 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:49.382685 (+ 445us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 08:34:49.395071 (+ 12386us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39357/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 08:34:49.395322 (+ 251us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51398: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39357/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":90}
W20260807 08:34:49.395545 17429 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51398: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39357/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 08:34:49.395895 17463 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:49.378451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:38007 (local address 127.0.0.1:51398)
0807 08:34:49.378949 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:49.378969 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:49.379176 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:49.379546 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:49.379550 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:49.379559 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 08:34:49.379711 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:49.379718 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:49.380674 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:49.380678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:49.381747 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:49.381757 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:49.382277 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:49.382281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:49.382386 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:49.382419 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 08:34:49.395336 (+ 12917us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39357/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 08:34:49.395635 (+ 299us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.174.62:38007: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39357/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":222,"threads_started":1}
I20260807 08:34:49.396697 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 17330
2026-08-07T08:34:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (381 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 08:34:49.446899 9912 mini_oidc.cc:152] Starting JWKS server
I20260807 08:34:49.448021 9912 webserver.cc:460] Webserver started at https://127.0.0.1:37201/ using document root <none> and password file <none>
I20260807 08:34:49.448122 9912 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 08:34:49.448352 9912 webserver.cc:460] Webserver started at http://127.0.0.1:38125/ using document root <none> and password file <none>
2026-08-07T08:34:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:49Z Disabled control of system clock
I20260807 08:34:49.462073 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:40183
--webserver_interface=127.9.174.62
--webserver_port=0
--builtin_ntp_servers=127.9.174.20:32929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:40183
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37201/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 08:34:49.597662 17479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:49.598042 17479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:49.598088 17479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:49.602458 17479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:49.602546 17479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:49.602563 17479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:49.602581 17479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:49.602597 17479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 08:34:49.602614 17479 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 08:34:49.602628 17479 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37201/jwks.json
I20260807 08:34:49.607867 17479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:32929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:40183
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:40183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37201/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:49.609350 17479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:49.611265 17479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:49.623442 17486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:49.623440 17487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:49.623433 17489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:49.623684 17479 server_base.cc:1034] running on GCE node
I20260807 08:34:49.624450 17479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:49.625731 17479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:49.626931 17479 hybrid_clock.cc:648] HybridClock initialized: now 1786091689626911 us; error 34 us; skew 500 ppm
I20260807 08:34:49.629215 17479 webserver.cc:460] Webserver started at http://127.9.174.62:40387/ using document root <none> and password file <none>
I20260807 08:34:49.629944 17479 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:49.630018 17479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:49.630270 17479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:49.632808 17479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "dc4eaa75eb394ed4bfd102712a4d0d18"
format_stamp: "Formatted at 2026-08-07 08:34:49 on dist-test-slave-h3ft"
server_key: "855b37172b64d2d50a8d37cb9c93a9d2"
server_key_iv: "ba03bf837cf18fe59d0e47db580b257f"
server_key_version: "encryptionkey@0"
I20260807 08:34:49.633459 17479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "dc4eaa75eb394ed4bfd102712a4d0d18"
format_stamp: "Formatted at 2026-08-07 08:34:49 on dist-test-slave-h3ft"
server_key: "855b37172b64d2d50a8d37cb9c93a9d2"
server_key_iv: "ba03bf837cf18fe59d0e47db580b257f"
server_key_version: "encryptionkey@0"
I20260807 08:34:49.637692 17479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 08:34:49.640731 17496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:49.642364 17479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 08:34:49.642549 17479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "dc4eaa75eb394ed4bfd102712a4d0d18"
format_stamp: "Formatted at 2026-08-07 08:34:49 on dist-test-slave-h3ft"
server_key: "855b37172b64d2d50a8d37cb9c93a9d2"
server_key_iv: "ba03bf837cf18fe59d0e47db580b257f"
server_key_version: "encryptionkey@0"
I20260807 08:34:49.642694 17479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:49.667286 17479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:49.668458 17479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:49.668740 17479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:49.677510 17479 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:40183
I20260807 08:34:49.677531 17569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:40183 every 8 connection(s)
I20260807 08:34:49.678711 17479 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:49.681449 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 17479
I20260807 08:34:49.681643 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:34:49.682039 9912 external_mini_cluster.cc:1428] Setting key af711d3d014ef8ff20a71de1b6b983f8
I20260807 08:34:49.683102 17570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:49.690801 17570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18: Bootstrap starting.
I20260807 08:34:49.693410 17575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:49.683770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33902 (local address 127.9.174.62:40183)
0807 08:34:49.684673 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:49.684693 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:49.684738 (+ 45us) server_negotiation.cc:407] Connection header received
0807 08:34:49.685582 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:49.685601 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:49.686093 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:49.686508 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:49.687109 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:49.688271 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:49.689325 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:49.689908 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:49.690113 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:49.691277 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:49.691282 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:49.691297 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:34:49.691791 (+ 494us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:49.691861 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:34:49.691885 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:49.692026 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:49.692159 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 08:34:49.692488 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":181,"threads_started":1}
I20260807 08:34:49.693774 17570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:49.694818 17570 log.cc:824] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:49.697316 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:34:49.697846 17570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18: No bootstrap required, opened a new log
I20260807 08:34:49.702828 17570 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4eaa75eb394ed4bfd102712a4d0d18" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40183 } }
I20260807 08:34:49.703294 17570 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:49.703380 17570 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc4eaa75eb394ed4bfd102712a4d0d18, State: Initialized, Role: FOLLOWER
I20260807 08:34:49.704020 17570 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4eaa75eb394ed4bfd102712a4d0d18" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40183 } }
I20260807 08:34:49.704219 17570 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:34:49.704332 17570 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:34:49.704475 17570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:49.706240 17570 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4eaa75eb394ed4bfd102712a4d0d18" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40183 } }
I20260807 08:34:49.706724 17570 leader_election.cc:302] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc4eaa75eb394ed4bfd102712a4d0d18; no voters:
I20260807 08:34:49.707288 17570 leader_election.cc:288] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:34:49.707475 17577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:49.707768 17577 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [term 1 LEADER]: Becoming Leader. State: Replica: dc4eaa75eb394ed4bfd102712a4d0d18, State: Running, Role: LEADER
I20260807 08:34:49.708153 17577 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4eaa75eb394ed4bfd102712a4d0d18" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40183 } }
I20260807 08:34:49.709028 17570 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:49.710282 17578 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc4eaa75eb394ed4bfd102712a4d0d18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4eaa75eb394ed4bfd102712a4d0d18" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40183 } } }
I20260807 08:34:49.710480 17578 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:49.710791 17579 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc4eaa75eb394ed4bfd102712a4d0d18. Latest consensus state: current_term: 1 leader_uuid: "dc4eaa75eb394ed4bfd102712a4d0d18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4eaa75eb394ed4bfd102712a4d0d18" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40183 } } }
I20260807 08:34:49.710891 17579 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:49.710891 17590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:49.714660 17590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:49.721853 17590 catalog_manager.cc:1347] Generated new cluster ID: 04d5bae0835243ccb7ff33652ce35a46
I20260807 08:34:49.721941 17590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:49.732133 17590 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:49.733335 17590 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:49.742708 17590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc4eaa75eb394ed4bfd102712a4d0d18: Generated new TSK 0
I20260807 08:34:49.743863 17590 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:34:49.767534 17575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:49.761497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33918 (local address 127.9.174.62:40183)
0807 08:34:49.761675 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:49.761681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:49.761774 (+ 93us) server_negotiation.cc:407] Connection header received
0807 08:34:49.761987 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:49.761991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:49.762058 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:49.762195 (+ 137us) server_negotiation.cc:226] Negotiated authn=JWT
0807 08:34:49.762462 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:49.763404 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:49.764381 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:49.764747 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:49.764849 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:49.765222 (+ 373us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 08:34:49.767052 (+ 1830us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 08:34:49.767332 (+ 280us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33918: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 08:34:49.767634 17575 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33918: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 08:34:49.769284 17611 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:49.761294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:40183 (local address 127.0.0.1:33918)
0807 08:34:49.761647 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:49.761662 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:49.761804 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:49.762214 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:49.762216 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:49.762224 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0807 08:34:49.762337 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:49.762342 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:49.763534 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:49.763537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:49.764255 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:49.764264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:49.764910 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:49.764913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:49.764980 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:49.765005 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 08:34:49.767250 (+ 2245us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 08:34:49.769048 (+ 1798us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.174.62:40183: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":1544,"thread_start_us":121,"threads_started":1}
I20260807 08:34:49.770040 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 17479
E20260807 08:34:49.789559 17612 webserver.cc:533] Webserver: sslize failed:
E20260807 08:34:49.789700 17612 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T08:34:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (378 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T08:34:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:49Z Disabled control of system clock
I20260807 08:34:49.837817 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:40437
--webserver_interface=127.9.174.62
--webserver_port=0
--builtin_ntp_servers=127.9.174.20:45411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:40437
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/testchainca.cert with env {}
W20260807 08:34:49.976044 17616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:49.976418 17616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:49.976462 17616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:49.980880 17616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:49.980965 17616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:49.980981 17616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:49.981004 17616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:49.981020 17616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 08:34:49.981063 17616 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 08:34:49.981099 17616 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 08:34:49.987402 17616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:40437
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:40437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:49.989045 17616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:49.990751 17616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:50.001502 17623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:50.001425 17624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:50.001425 17628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:50.001639 17616 server_base.cc:1034] running on GCE node
I20260807 08:34:50.002693 17616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:50.004638 17616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:50.005864 17616 hybrid_clock.cc:648] HybridClock initialized: now 1786091690005846 us; error 44 us; skew 500 ppm
I20260807 08:34:50.008859 17616 webserver.cc:460] Webserver started at http://127.9.174.62:39093/ using document root <none> and password file <none>
I20260807 08:34:50.009774 17616 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:50.009896 17616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:50.010212 17616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:50.013274 17616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "c555703d803f4cefb58d7d9a44f5df96"
format_stamp: "Formatted at 2026-08-07 08:34:50 on dist-test-slave-h3ft"
server_key: "a33b5d961156611f585d2f9566dba243"
server_key_iv: "054d8ed606c2d54216a9097eb08448f1"
server_key_version: "encryptionkey@0"
I20260807 08:34:50.014034 17616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "c555703d803f4cefb58d7d9a44f5df96"
format_stamp: "Formatted at 2026-08-07 08:34:50 on dist-test-slave-h3ft"
server_key: "a33b5d961156611f585d2f9566dba243"
server_key_iv: "054d8ed606c2d54216a9097eb08448f1"
server_key_version: "encryptionkey@0"
I20260807 08:34:50.019150 17616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 08:34:50.022503 17634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:50.024232 17616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:34:50.024434 17616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "c555703d803f4cefb58d7d9a44f5df96"
format_stamp: "Formatted at 2026-08-07 08:34:50 on dist-test-slave-h3ft"
server_key: "a33b5d961156611f585d2f9566dba243"
server_key_iv: "054d8ed606c2d54216a9097eb08448f1"
server_key_version: "encryptionkey@0"
I20260807 08:34:50.024621 17616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:50.059924 17616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:50.061051 17616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:50.061339 17616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:50.070672 17616 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:40437
I20260807 08:34:50.070735 17710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:40437 every 8 connection(s)
I20260807 08:34:50.072021 17616 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:50.076473 17711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:50.077173 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 17616
I20260807 08:34:50.077343 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:34:50.077771 9912 external_mini_cluster.cc:1428] Setting key 891177bc3b7c4b35727705bf4cf18869
I20260807 08:34:50.085513 17711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96: Bootstrap starting.
I20260807 08:34:50.089145 17711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:50.090026 17715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:50.079631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38836 (local address 127.9.174.62:40437)
0807 08:34:50.080508 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:50.080529 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:50.080579 (+ 50us) server_negotiation.cc:407] Connection header received
0807 08:34:50.081382 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:50.081406 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:50.081945 (+ 539us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:50.082308 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:50.082823 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:50.084063 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:50.085436 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:50.086077 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:50.086218 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:50.087380 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:50.087386 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:50.087403 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:34:50.087910 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:50.087983 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:34:50.088037 (+ 54us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:50.088129 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:50.088368 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 08:34:50.088769 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":378,"thread_start_us":200,"threads_started":1}
I20260807 08:34:50.090150 17711 log.cc:824] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:50.092993 17711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96: No bootstrap required, opened a new log
I20260807 08:34:50.093806 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:34:50.098031 17711 raft_consensus.cc:348] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c555703d803f4cefb58d7d9a44f5df96" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40437 } }
I20260807 08:34:50.098392 17711 raft_consensus.cc:374] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:50.098477 17711 raft_consensus.cc:729] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c555703d803f4cefb58d7d9a44f5df96, State: Initialized, Role: FOLLOWER
I20260807 08:34:50.099175 17711 consensus_queue.cc:260] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c555703d803f4cefb58d7d9a44f5df96" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40437 } }
I20260807 08:34:50.099416 17711 raft_consensus.cc:388] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:34:50.099540 17711 raft_consensus.cc:482] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:34:50.099712 17711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:50.101212 17711 raft_consensus.cc:504] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c555703d803f4cefb58d7d9a44f5df96" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40437 } }
I20260807 08:34:50.101747 17711 leader_election.cc:302] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c555703d803f4cefb58d7d9a44f5df96; no voters:
I20260807 08:34:50.102212 17711 leader_election.cc:288] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:34:50.102440 17717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:50.102716 17717 raft_consensus.cc:686] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [term 1 LEADER]: Becoming Leader. State: Replica: c555703d803f4cefb58d7d9a44f5df96, State: Running, Role: LEADER
I20260807 08:34:50.103194 17717 consensus_queue.cc:237] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c555703d803f4cefb58d7d9a44f5df96" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40437 } }
I20260807 08:34:50.104085 17711 sys_catalog.cc:564] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:50.105541 17718 sys_catalog.cc:455] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c555703d803f4cefb58d7d9a44f5df96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c555703d803f4cefb58d7d9a44f5df96" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40437 } } }
I20260807 08:34:50.105607 17719 sys_catalog.cc:455] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c555703d803f4cefb58d7d9a44f5df96. Latest consensus state: current_term: 1 leader_uuid: "c555703d803f4cefb58d7d9a44f5df96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c555703d803f4cefb58d7d9a44f5df96" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40437 } } }
I20260807 08:34:50.105710 17718 sys_catalog.cc:458] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:50.105726 17719 sys_catalog.cc:458] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:50.106266 17728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:50.110263 17728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:50.118117 17728 catalog_manager.cc:1347] Generated new cluster ID: 992066a38076470e937779575cb04943
I20260807 08:34:50.118242 17728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:50.133270 17728 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:50.134519 17728 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:50.142299 17728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c555703d803f4cefb58d7d9a44f5df96: Generated new TSK 0
I20260807 08:34:50.143292 17728 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:34:50.649394 17715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:50.154058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38840 (local address 127.9.174.62:40437)
0807 08:34:50.154244 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:50.154252 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:50.154416 (+ 164us) server_negotiation.cc:407] Connection header received
0807 08:34:50.154664 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:50.154668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:50.154730 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:50.154846 (+ 116us) server_negotiation.cc:226] Negotiated authn=JWT
0807 08:34:50.155200 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:50.155833 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:50.157153 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:50.157401 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:50.157499 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:50.157887 (+ 388us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 08:34:50.648753 (+490866us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 08:34:50.649205 (+ 452us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38840: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 08:34:50.649540 17715 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38840: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 08:34:50.649748 17749 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:50.153950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:40437 (local address 127.0.0.1:38840)
0807 08:34:50.154305 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:50.154321 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:50.154475 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:50.154887 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:50.154893 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:50.154903 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0807 08:34:50.155075 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:50.155082 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:50.155966 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:50.155970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:50.157017 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:50.157028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:50.157547 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:50.157550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:50.157653 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:50.157685 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 08:34:50.649130 (+491445us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 08:34:50.649390 (+ 260us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.174.62:40437: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":139,"threads_started":1}
I20260807 08:34:50.650759 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 17616
2026-08-07T08:34:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (855 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (153 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (158 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:34:51 dist-test-slave-h3ft krb5kdc[17765](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:34:51 dist-test-slave-h3ft krb5kdc[17765](info): set up 2 sockets
Aug 07 08:34:51 dist-test-slave-h3ft krb5kdc[17765](info): commencing operation
krb5kdc: starting...
W20260807 08:34:52.285986 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:34:52 dist-test-slave-h3ft krb5kdc[17765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:34:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:34:52.446489 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:38291
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:42967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:38291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:52.583197 17780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:52.583550 17780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:52.583633 17780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:52.587981 17780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:52.588104 17780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:52.588147 17780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:52.588188 17780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:52.588234 17780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:52.593884 17780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:42967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:38291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:38291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:52.595184 17780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:52.596376 17780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:52.607065 17789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:52.607065 17791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:52.607121 17794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:52.607477 17780 server_base.cc:1034] running on GCE node
I20260807 08:34:52.608261 17780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:52.609609 17780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:52.610816 17780 hybrid_clock.cc:648] HybridClock initialized: now 1786091692610786 us; error 59 us; skew 500 ppm
Aug 07 08:34:52 dist-test-slave-h3ft krb5kdc[17765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091692, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:52.614156 17780 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:34:52.615389 17780 webserver.cc:460] Webserver started at http://127.9.174.62:43911/ using document root <none> and password file <none>
I20260807 08:34:52.616072 17780 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:52.616143 17780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:52.616348 17780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:52.618556 17780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "1beb0c15b40e42f8bcaf21a582b6fdf8"
format_stamp: "Formatted at 2026-08-07 08:34:52 on dist-test-slave-h3ft"
server_key: "c837dc30bab50b1718875a9ef4d9e521"
server_key_iv: "09a0c51cb380f32e5f431f8f513c8804"
server_key_version: "encryptionkey@0"
I20260807 08:34:52.619189 17780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "1beb0c15b40e42f8bcaf21a582b6fdf8"
format_stamp: "Formatted at 2026-08-07 08:34:52 on dist-test-slave-h3ft"
server_key: "c837dc30bab50b1718875a9ef4d9e521"
server_key_iv: "09a0c51cb380f32e5f431f8f513c8804"
server_key_version: "encryptionkey@0"
I20260807 08:34:52.623921 17780 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:34:52.627243 17803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:52.628935 17780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:34:52.629073 17780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "1beb0c15b40e42f8bcaf21a582b6fdf8"
format_stamp: "Formatted at 2026-08-07 08:34:52 on dist-test-slave-h3ft"
server_key: "c837dc30bab50b1718875a9ef4d9e521"
server_key_iv: "09a0c51cb380f32e5f431f8f513c8804"
server_key_version: "encryptionkey@0"
I20260807 08:34:52.629211 17780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:52.643179 17780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:52.646878 17780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:52.647173 17780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:52.656241 17780 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:38291
I20260807 08:34:52.656335 17880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:38291 every 8 connection(s)
I20260807 08:34:52.657470 17780 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:52.661298 17882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:52.667156 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 17780
I20260807 08:34:52.667300 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:34:52.667659 9912 external_mini_cluster.cc:1428] Setting key e21df61a909f213d32ad70b4def3cf0b
I20260807 08:34:52.668865 17882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8: Bootstrap starting.
I20260807 08:34:52.671896 17882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:52.672851 17882 log.cc:824] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:52.675271 17882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8: No bootstrap required, opened a new log
Aug 07 08:34:52 dist-test-slave-h3ft krb5kdc[17765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:52.680151 17882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1beb0c15b40e42f8bcaf21a582b6fdf8" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38291 } }
I20260807 08:34:52.680523 17882 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:52.680594 17882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1beb0c15b40e42f8bcaf21a582b6fdf8, State: Initialized, Role: FOLLOWER
I20260807 08:34:52.681146 17882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1beb0c15b40e42f8bcaf21a582b6fdf8" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38291 } }
I20260807 08:34:52.681363 17882 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:34:52.681468 17882 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:34:52.681617 17882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:52.682879 17882 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1beb0c15b40e42f8bcaf21a582b6fdf8" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38291 } }
I20260807 08:34:52.683398 17882 leader_election.cc:302] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1beb0c15b40e42f8bcaf21a582b6fdf8; no voters:
I20260807 08:34:52.683893 17882 leader_election.cc:288] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:34:52.684077 17887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:52.684423 17887 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [term 1 LEADER]: Becoming Leader. State: Replica: 1beb0c15b40e42f8bcaf21a582b6fdf8, State: Running, Role: LEADER
I20260807 08:34:52.684872 17887 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1beb0c15b40e42f8bcaf21a582b6fdf8" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38291 } }
I20260807 08:34:52.685772 17882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:52.687414 17888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1beb0c15b40e42f8bcaf21a582b6fdf8. Latest consensus state: current_term: 1 leader_uuid: "1beb0c15b40e42f8bcaf21a582b6fdf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1beb0c15b40e42f8bcaf21a582b6fdf8" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38291 } } }
I20260807 08:34:52.687574 17888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:52.687786 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:52.669401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.9.174.62:38291)
0807 08:34:52.670101 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:52.670123 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:52.670194 (+ 71us) server_negotiation.cc:407] Connection header received
0807 08:34:52.671280 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:52.671306 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:52.671775 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:52.672218 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:52.673259 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:52.674392 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:52.675438 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:52.676006 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:52.676211 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:52.679907 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:52.679914 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:52.679936 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:52.679978 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:52.682521 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:52.683292 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:52.683299 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:52.683305 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:52.683450 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:52.683878 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:52.683883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:52.683888 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:52.684612 (+ 724us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:52.684901 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:52.685846 (+ 945us) server_negotiation.cc:299] Negotiation successful
0807 08:34:52.686272 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":194,"threads_started":1}
I20260807 08:34:52.689517 17889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1beb0c15b40e42f8bcaf21a582b6fdf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1beb0c15b40e42f8bcaf21a582b6fdf8" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38291 } } }
I20260807 08:34:52.689654 17889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8 [sys.catalog]: This master's current role is: LEADER
W20260807 08:34:52.692760 17908 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:34:52.692873 17908 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:34:52.692955 17899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:52.694003 17899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:52.701514 17899 catalog_manager.cc:1347] Generated new cluster ID: e03d957b5edb4d67850e9ddd9c739ed1
I20260807 08:34:52.701630 17899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:52.719038 17899 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:52.720371 17899 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:52.726382 17899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1beb0c15b40e42f8bcaf21a582b6fdf8: Generated new TSK 0
I20260807 08:34:52.727548 17899 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:34:52.778703 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:38291
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:42967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:52.930447 17915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:52.930846 17915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:52.930939 17915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:52.935484 17915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:52.935580 17915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:52.935674 17915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:34:52.941591 17915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:42967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:38291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:52.943073 17915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:52.944410 17915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:52.957233 17925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:52.957224 17924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:52.957233 17929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:52.957716 17915 server_base.cc:1034] running on GCE node
I20260807 08:34:52.958230 17915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:52.958976 17915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:52.960258 17915 hybrid_clock.cc:648] HybridClock initialized: now 1786091692960235 us; error 36 us; skew 500 ppm
Aug 07 08:34:52 dist-test-slave-h3ft krb5kdc[17765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091692, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:52.963840 17915 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:34:52.965085 17915 webserver.cc:460] Webserver started at http://127.9.174.1:37889/ using document root <none> and password file <none>
I20260807 08:34:52.965781 17915 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:52.965857 17915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:52.966087 17915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:52.969200 17915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "8bb9875031494d18a8a029bbfa73ee16"
format_stamp: "Formatted at 2026-08-07 08:34:52 on dist-test-slave-h3ft"
server_key: "186b3113e4b20966d78914d63f60b28b"
server_key_iv: "244c854e3a259363989c711616473cdc"
server_key_version: "encryptionkey@0"
I20260807 08:34:52.969818 17915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "8bb9875031494d18a8a029bbfa73ee16"
format_stamp: "Formatted at 2026-08-07 08:34:52 on dist-test-slave-h3ft"
server_key: "186b3113e4b20966d78914d63f60b28b"
server_key_iv: "244c854e3a259363989c711616473cdc"
server_key_version: "encryptionkey@0"
I20260807 08:34:52.974732 17915 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:34:52.978235 17937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:52.980216 17915 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 08:34:52.980387 17915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "8bb9875031494d18a8a029bbfa73ee16"
format_stamp: "Formatted at 2026-08-07 08:34:52 on dist-test-slave-h3ft"
server_key: "186b3113e4b20966d78914d63f60b28b"
server_key_iv: "244c854e3a259363989c711616473cdc"
server_key_version: "encryptionkey@0"
I20260807 08:34:52.980592 17915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:53.018713 17915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:53.023334 17915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:53.023612 17915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:53.024430 17915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:53.025525 17915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:53.025605 17915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:53.025691 17915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:53.025734 17915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:53.038786 17915 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:37373
I20260807 08:34:53.038882 18091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:37373 every 8 connection(s)
I20260807 08:34:53.040118 17915 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:34:53.040360 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 17915
I20260807 08:34:53.040522 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:34:53.040987 9912 external_mini_cluster.cc:1428] Setting key 32411b39ce98234cfda33efc154a98a1
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[17765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091692, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
I20260807 08:34:53.057822 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:53.042880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:41563 (local address 127.9.174.62:38291)
0807 08:34:53.043079 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:53.043085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:53.044721 (+ 1636us) server_negotiation.cc:407] Connection header received
0807 08:34:53.046517 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:53.046521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:53.046587 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:53.046694 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:53.048542 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.049135 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.050451 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.050701 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.050824 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:53.053754 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:53.053757 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:53.053760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:53.053788 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:53.055499 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:53.056068 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:53.056103 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:53.056107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:53.056223 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:53.056617 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:53.056620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:53.056622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:53.056828 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:53.056941 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:53.057486 (+ 545us) server_negotiation.cc:299] Negotiation successful
0807 08:34:53.057629 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
I20260807 08:34:53.058743 18096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:53.043618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:38291 (local address 127.9.174.1:41563)
0807 08:34:53.044473 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:53.044536 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:53.046201 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:53.046973 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:53.046983 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:53.047670 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:53.048362 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:53.048377 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.049281 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.049287 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:53.050322 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:53.050333 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.050880 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.050885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:53.051123 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:53.051806 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:53.051824 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:53.053542 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:53.055639 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:53.055646 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:53.055665 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:53.055949 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:53.056340 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:53.056343 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:53.056345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:53.056503 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:53.056942 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:53.056950 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:53.057229 (+ 279us) client_negotiation.cc:769] Sending connection context
0807 08:34:53.057428 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 08:34:53.057795 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":290,"threads_started":1}
I20260807 08:34:53.060496 18092 heartbeater.cc:347] Connected to a master server at 127.9.174.62:38291
I20260807 08:34:53.060952 18092 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:53.061761 18092 heartbeater.cc:511] Master 127.9.174.62:38291 requested a full tablet report, sending...
I20260807 08:34:53.064184 17825 ts_manager.cc:194] Registered new tserver with Master: 8bb9875031494d18a8a029bbfa73ee16 (127.9.174.1:37373)
I20260807 08:34:53.066303 17825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:41563
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:34:53.083853 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:38291
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:42967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:53.234795 18100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:53.235301 18100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:53.235404 18100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:53.240502 18100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:53.240629 18100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:53.240744 18100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:34:53.247680 18100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:42967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:38291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:53.249308 18100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:53.250670 18100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:53.263255 18111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:53.263255 18107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:53.263381 18108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:53.264034 18100 server_base.cc:1034] running on GCE node
I20260807 08:34:53.264660 18100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:53.265484 18100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:53.266680 18100 hybrid_clock.cc:648] HybridClock initialized: now 1786091693266638 us; error 61 us; skew 500 ppm
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[17765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091693, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:53.275656 18100 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:34:53.276891 18100 webserver.cc:460] Webserver started at http://127.9.174.2:37321/ using document root <none> and password file <none>
I20260807 08:34:53.277613 18100 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:53.277721 18100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:53.277990 18100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:53.280512 18100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "c71f2504b9a84438a8ac6c7afd7e7ff8"
format_stamp: "Formatted at 2026-08-07 08:34:53 on dist-test-slave-h3ft"
server_key: "0b96dbb469942ca37423530371b2a62b"
server_key_iv: "bcb31c2b1f70c7161372264448b2b4e9"
server_key_version: "encryptionkey@0"
I20260807 08:34:53.281169 18100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "c71f2504b9a84438a8ac6c7afd7e7ff8"
format_stamp: "Formatted at 2026-08-07 08:34:53 on dist-test-slave-h3ft"
server_key: "0b96dbb469942ca37423530371b2a62b"
server_key_iv: "bcb31c2b1f70c7161372264448b2b4e9"
server_key_version: "encryptionkey@0"
I20260807 08:34:53.286198 18100 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:34:53.290184 18119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:53.292194 18100 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:34:53.292378 18100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "c71f2504b9a84438a8ac6c7afd7e7ff8"
format_stamp: "Formatted at 2026-08-07 08:34:53 on dist-test-slave-h3ft"
server_key: "0b96dbb469942ca37423530371b2a62b"
server_key_iv: "bcb31c2b1f70c7161372264448b2b4e9"
server_key_version: "encryptionkey@0"
I20260807 08:34:53.292531 18100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:53.323760 18100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:53.327853 18100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:53.328115 18100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:53.328991 18100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:53.330113 18100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:53.330197 18100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:53.330286 18100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:53.330333 18100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:53.342464 18100 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:44249
I20260807 08:34:53.342563 18285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:44249 every 8 connection(s)
I20260807 08:34:53.343618 18100 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:34:53.345682 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 18100
I20260807 08:34:53.345837 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:34:53.346186 9912 external_mini_cluster.cc:1428] Setting key 21bcf19e43be06895e0979295b988c01
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[17765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091693, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
I20260807 08:34:53.364059 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:53.347174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:49887 (local address 127.9.174.62:38291)
0807 08:34:53.347403 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:53.347412 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:53.348227 (+ 815us) server_negotiation.cc:407] Connection header received
0807 08:34:53.349865 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:53.349869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:53.349928 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:53.350009 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:53.351669 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.352318 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.354334 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.354604 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.354706 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:53.358756 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:53.358760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:53.358763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:53.358799 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:53.360732 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:53.361629 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:53.361633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:53.361635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:53.361722 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:53.362143 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:53.362147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:53.362149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:53.362434 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:53.362608 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:53.363608 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0807 08:34:53.363880 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 08:34:53.365197 18291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:53.347101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:38291 (local address 127.9.174.2:49887)
0807 08:34:53.347968 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:53.348051 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:53.349602 (+ 1551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:53.350215 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:53.350223 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:53.350638 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:53.351451 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:53.351475 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.352513 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.352516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:53.354164 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:53.354182 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.354829 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.354836 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:53.355157 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:53.356108 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:53.356139 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:53.358481 (+ 2342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:53.360900 (+ 2419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:53.360909 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:53.360939 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:53.361490 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:53.361826 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:53.361829 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:53.361831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:53.362029 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:53.362609 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:53.362621 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:53.363042 (+ 421us) client_negotiation.cc:769] Sending connection context
0807 08:34:53.363351 (+ 309us) client_negotiation.cc:240] Negotiation successful
0807 08:34:53.363866 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":224,"threads_started":1}
I20260807 08:34:53.366840 18288 heartbeater.cc:347] Connected to a master server at 127.9.174.62:38291
I20260807 08:34:53.367202 18288 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:53.367892 18288 heartbeater.cc:511] Master 127.9.174.62:38291 requested a full tablet report, sending...
I20260807 08:34:53.369280 17825 ts_manager.cc:194] Registered new tserver with Master: c71f2504b9a84438a8ac6c7afd7e7ff8 (127.9.174.2:44249)
I20260807 08:34:53.370023 17825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:49887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:34:53.383781 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:38291
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:42967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:53.527333 18295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:53.527729 18295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:53.527820 18295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:53.532267 18295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:53.532415 18295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:53.532531 18295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:34:53.540488 18295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:42967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:38291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:53.541923 18295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:53.543295 18295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:53.555610 18302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:53.555610 18303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:53.555642 18306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:53.555951 18295 server_base.cc:1034] running on GCE node
I20260807 08:34:53.556558 18295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:53.557421 18295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:53.558677 18295 hybrid_clock.cc:648] HybridClock initialized: now 1786091693558656 us; error 36 us; skew 500 ppm
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[17765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091693, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:53.562218 18295 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:34:53.563441 18295 webserver.cc:460] Webserver started at http://127.9.174.3:36109/ using document root <none> and password file <none>
I20260807 08:34:53.564086 18295 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:53.564194 18295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:53.564453 18295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:53.566802 18295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "bc69c7fc340d41a4aafc358a2c19097b"
format_stamp: "Formatted at 2026-08-07 08:34:53 on dist-test-slave-h3ft"
server_key: "8bc8b53ece0994a33a9f05e6c44b8789"
server_key_iv: "d875b01ce824008d52af94cf85addee1"
server_key_version: "encryptionkey@0"
I20260807 08:34:53.567502 18295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "bc69c7fc340d41a4aafc358a2c19097b"
format_stamp: "Formatted at 2026-08-07 08:34:53 on dist-test-slave-h3ft"
server_key: "8bc8b53ece0994a33a9f05e6c44b8789"
server_key_iv: "d875b01ce824008d52af94cf85addee1"
server_key_version: "encryptionkey@0"
I20260807 08:34:53.571949 18295 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:34:53.575318 18314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:53.577060 18295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:34:53.577271 18295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "bc69c7fc340d41a4aafc358a2c19097b"
format_stamp: "Formatted at 2026-08-07 08:34:53 on dist-test-slave-h3ft"
server_key: "8bc8b53ece0994a33a9f05e6c44b8789"
server_key_iv: "d875b01ce824008d52af94cf85addee1"
server_key_version: "encryptionkey@0"
I20260807 08:34:53.577456 18295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:53.601049 18295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:53.605799 18295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:53.606086 18295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:53.606948 18295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:53.608160 18295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:53.608213 18295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:53.608264 18295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:53.608280 18295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:53.620169 18295 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:44757
I20260807 08:34:53.620194 18480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:44757 every 8 connection(s)
I20260807 08:34:53.621344 18295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:34:53.624342 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 18295
I20260807 08:34:53.624495 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:34:53.624894 9912 external_mini_cluster.cc:1428] Setting key a1e29f14e423be8910b52fccee61ada3
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[17765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091693, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:34:53.639504 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:53.623760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:48541 (local address 127.9.174.62:38291)
0807 08:34:53.624008 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:53.624013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:53.625355 (+ 1342us) server_negotiation.cc:407] Connection header received
0807 08:34:53.627418 (+ 2063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:53.627422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:53.627491 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:53.627608 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:53.629297 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.630195 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.631887 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.632152 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.632279 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:53.635463 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:53.635466 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:53.635468 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:53.635496 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:53.636921 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:53.637515 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:53.637519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:53.637521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:53.637622 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:53.638041 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:53.638045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:53.638046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:53.638266 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:53.638385 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:53.639162 (+ 777us) server_negotiation.cc:299] Negotiation successful
0807 08:34:53.639308 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 08:34:53.640720 18484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:53.624362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:38291 (local address 127.9.174.3:48541)
0807 08:34:53.625149 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:53.625205 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:53.627095 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:53.627831 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:53.627840 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:53.628328 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:53.629085 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:53.629115 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.630514 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.630520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:53.631734 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:53.631746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.632346 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.632350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:53.632591 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:53.633373 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:53.633393 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:53.635249 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:53.637053 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:53.637069 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:53.637090 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:53.637397 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:53.637753 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:53.637757 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:53.637759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:53.637928 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:53.638385 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:53.638393 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:53.638829 (+ 436us) client_negotiation.cc:769] Sending connection context
0807 08:34:53.639109 (+ 280us) client_negotiation.cc:240] Negotiation successful
0807 08:34:53.639530 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":210,"threads_started":1}
I20260807 08:34:53.642045 18482 heartbeater.cc:347] Connected to a master server at 127.9.174.62:38291
I20260807 08:34:53.642387 18482 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:53.643139 18482 heartbeater.cc:511] Master 127.9.174.62:38291 requested a full tablet report, sending...
I20260807 08:34:53.644443 17825 ts_manager.cc:194] Registered new tserver with Master: bc69c7fc340d41a4aafc358a2c19097b (127.9.174.3:44757)
I20260807 08:34:53.645135 17825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:48541
I20260807 08:34:53.652546 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/test-admin.keytab.
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[17765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:53.676220 9912 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 08:34:53.680322 18490 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:53.678248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41344 (local address 127.9.174.1:37373)
0807 08:34:53.678650 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:53.678659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:53.678687 (+ 28us) server_negotiation.cc:407] Connection header received
0807 08:34:53.678751 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:53.678757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:53.679118 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:53.679272 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:53.680091 (+ 819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41344: BlockingRecv error: recv got EOF from 127.0.0.1:41344 (error 108)
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":181,"threads_started":1}
W20260807 08:34:53.680385 9912 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/krb5cc' of type FILE
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[17765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:53.681704 9912 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[17765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:53.682817 9912 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[17765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.1@KRBTEST.COM
I20260807 08:34:53.695272 18490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:53.684221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41348 (local address 127.9.174.1:37373)
0807 08:34:53.684428 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:53.684435 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:53.684460 (+ 25us) server_negotiation.cc:407] Connection header received
0807 08:34:53.684565 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:53.684568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:53.684619 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:53.684695 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:53.685286 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.686317 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.687471 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.687969 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.688072 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:53.690793 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:53.690805 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:53.690824 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:53.690854 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:53.692908 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:53.693490 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:53.693497 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:53.693499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:53.693620 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:53.694018 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:53.694022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:53.694023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:53.694353 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:53.694637 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:53.694864 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 08:34:53.695089 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 08:34:53.700241 18490 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:53.699091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41354 (local address 127.9.174.1:37373)
0807 08:34:53.699247 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:53.699251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:53.699284 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:34:53.699450 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:53.699453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:53.699509 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:53.699583 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:53.700087 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41354: BlockingRecv error: recv got EOF from 127.0.0.1:41354 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 08:34:53.700242 9912 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/krb5cc' of type FILE
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[17765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:53.701546 9912 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[17765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:53.702554 9912 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[17765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.1@KRBTEST.COM
I20260807 08:34:53.713168 18490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:53.703725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41368 (local address 127.9.174.1:37373)
0807 08:34:53.703943 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:53.703947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:53.703964 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:34:53.704053 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:53.704056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:53.704103 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:53.704185 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:53.704726 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.705348 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.706198 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.706441 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.706528 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:53.709305 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:53.709309 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:53.709311 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:53.709352 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:53.710937 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:53.711596 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:53.711599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:53.711601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:53.711692 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:53.712092 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:53.712096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:53.712097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:53.712317 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:53.712462 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:53.712716 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 08:34:53.712933 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 08:34:53.716902 18490 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:53.715809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41376 (local address 127.9.174.1:37373)
0807 08:34:53.715938 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:53.715943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:53.716000 (+ 57us) server_negotiation.cc:407] Connection header received
0807 08:34:53.716147 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:53.716149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:53.716191 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:53.716261 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:53.716748 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41376: BlockingRecv error: recv got EOF from 127.0.0.1:41376 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 08:34:53.716974 9912 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091655732880-9912-0/krb5kdc/krb5cc' of type FILE
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[17765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:53.718297 9912 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[17765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:53.719359 9912 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[17765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.1@KRBTEST.COM
I20260807 08:34:53.729454 18490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:53.720385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41392 (local address 127.9.174.1:37373)
0807 08:34:53.720518 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:53.720522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:53.720555 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:34:53.720673 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:53.720676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:53.720721 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:53.720804 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:53.721347 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.721894 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.722937 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:53.723284 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:53.723386 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:53.726011 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:53.726015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:53.726017 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:53.726054 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:53.727684 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:53.728113 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:53.728116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:53.728118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:53.728210 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:53.728505 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:53.728508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:53.728509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:53.728711 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:53.728856 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:53.729117 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 08:34:53.729254 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:34:53.730837 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 17915
I20260807 08:34:53.747282 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 18100
I20260807 08:34:53.770143 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 18295
I20260807 08:34:53.789219 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 17780
2026-08-07T08:34:53Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2829 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[18508](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[18508](info): set up 2 sockets
Aug 07 08:34:53 dist-test-slave-h3ft krb5kdc[18508](info): commencing operation
krb5kdc: starting...
W20260807 08:34:55.117959 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:34:55 dist-test-slave-h3ft krb5kdc[18508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:34:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:55Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.62.keytab.
Entry for principal oryx/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.62.keytab.
I20260807 08:34:55.276837 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:42359
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.62.keytab
--principal=oryx/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:42359
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:55.414783 18528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:55.415215 18528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:55.415297 18528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:55.419425 18528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:55.419518 18528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:55.419567 18528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:55.419625 18528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:55.419663 18528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:55.425411 18528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:42359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.62.keytab
--principal=oryx/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:42359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:55.426846 18528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:55.428159 18528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:55.438364 18538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:55.438407 18541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:55.438421 18537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:55.438602 18528 server_base.cc:1034] running on GCE node
I20260807 08:34:55.439550 18528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:55.440851 18528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:55.442049 18528 hybrid_clock.cc:648] HybridClock initialized: now 1786091695442001 us; error 39 us; skew 500 ppm
Aug 07 08:34:55 dist-test-slave-h3ft krb5kdc[18508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091695, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:55.445370 18528 init.cc:377] Logged in from keytab as oryx/127.9.174.62@KRBTEST.COM (short username oryx)
I20260807 08:34:55.446532 18528 webserver.cc:460] Webserver started at http://127.9.174.62:35709/ using document root <none> and password file <none>
I20260807 08:34:55.447299 18528 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:55.447368 18528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:55.447671 18528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:55.450314 18528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "5c7b45d28ba243aa8679d3071c58d4c9"
format_stamp: "Formatted at 2026-08-07 08:34:55 on dist-test-slave-h3ft"
server_key: "8e507c8da7e7b7e6ec9e6810fdd3b61f"
server_key_iv: "8775ebc561d0c092de20fab99fb5f87b"
server_key_version: "encryptionkey@0"
I20260807 08:34:55.451074 18528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "5c7b45d28ba243aa8679d3071c58d4c9"
format_stamp: "Formatted at 2026-08-07 08:34:55 on dist-test-slave-h3ft"
server_key: "8e507c8da7e7b7e6ec9e6810fdd3b61f"
server_key_iv: "8775ebc561d0c092de20fab99fb5f87b"
server_key_version: "encryptionkey@0"
I20260807 08:34:55.455808 18528 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 08:34:55.458880 18548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:55.460469 18528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:34:55.460634 18528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "5c7b45d28ba243aa8679d3071c58d4c9"
format_stamp: "Formatted at 2026-08-07 08:34:55 on dist-test-slave-h3ft"
server_key: "8e507c8da7e7b7e6ec9e6810fdd3b61f"
server_key_iv: "8775ebc561d0c092de20fab99fb5f87b"
server_key_version: "encryptionkey@0"
I20260807 08:34:55.460762 18528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:55.474539 18528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:55.478551 18528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:55.478793 18528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:55.487833 18528 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:42359
I20260807 08:34:55.488190 18625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:42359 every 8 connection(s)
I20260807 08:34:55.489056 18528 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:55.492538 18627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:55.496803 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 18528
I20260807 08:34:55.496981 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:34:55.497300 9912 external_mini_cluster.cc:1428] Setting key a47a56a78dcd9dccc6b4423ad7f99c35
I20260807 08:34:55.499758 18627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9: Bootstrap starting.
I20260807 08:34:55.502753 18627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:55.503829 18627 log.cc:824] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:55.506177 18627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9: No bootstrap required, opened a new log
Aug 07 08:34:55 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM
I20260807 08:34:55.510926 18627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } }
I20260807 08:34:55.511297 18627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:55.511444 18627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7b45d28ba243aa8679d3071c58d4c9, State: Initialized, Role: FOLLOWER
I20260807 08:34:55.512048 18627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } }
I20260807 08:34:55.512254 18627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:34:55.512369 18627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:34:55.512496 18627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:55.513886 18627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } }
I20260807 08:34:55.514225 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:55.499114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34726 (local address 127.9.174.62:42359)
0807 08:34:55.499603 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:55.499621 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:55.499662 (+ 41us) server_negotiation.cc:407] Connection header received
0807 08:34:55.500396 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:55.500415 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:55.500791 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:55.501101 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:55.502105 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:55.503293 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:55.504495 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:55.504998 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:55.505191 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:55.508193 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:55.508199 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:55.508215 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:55.508247 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:55.510314 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:55.510871 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:55.510877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:55.510883 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:55.511065 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:55.511402 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:55.511407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:55.511410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:55.512043 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:55.512294 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:55.512665 (+ 371us) server_negotiation.cc:299] Negotiation successful
0807 08:34:55.513029 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":160,"threads_started":1}
I20260807 08:34:55.514300 18627 leader_election.cc:302] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c7b45d28ba243aa8679d3071c58d4c9; no voters:
I20260807 08:34:55.514714 18627 leader_election.cc:288] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:34:55.514844 18634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:55.515120 18634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [term 1 LEADER]: Becoming Leader. State: Replica: 5c7b45d28ba243aa8679d3071c58d4c9, State: Running, Role: LEADER
I20260807 08:34:55.515496 18634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } }
I20260807 08:34:55.516250 18627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:55.517516 18635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } }
I20260807 08:34:55.517665 18635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:55.518102 18636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c7b45d28ba243aa8679d3071c58d4c9. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } }
I20260807 08:34:55.518229 18636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:55.518102 18649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:55.521950 18649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:55.528345 18649 catalog_manager.cc:1347] Generated new cluster ID: 52ca67c8727643308ba3e9a056ffe10c
I20260807 08:34:55.528442 18649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:55.541402 18649 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:55.542537 18649 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:55.553587 18649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9: Generated new TSK 0
I20260807 08:34:55.554627 18649 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.1.keytab.
Entry for principal oryx/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.1.keytab.
I20260807 08:34:55.595372 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42359
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.1.keytab
--principal=oryx/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:55.741669 18667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:55.742040 18667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:55.742090 18667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:55.746483 18667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:55.746590 18667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:55.746658 18667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:34:55.753566 18667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.1.keytab
--principal=oryx/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:55.755242 18667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:55.756604 18667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:55.769170 18676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:55.769285 18674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:55.769192 18673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:55.769608 18667 server_base.cc:1034] running on GCE node
I20260807 08:34:55.770092 18667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:55.770903 18667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:55.772202 18667 hybrid_clock.cc:648] HybridClock initialized: now 1786091695772179 us; error 34 us; skew 500 ppm
Aug 07 08:34:55 dist-test-slave-h3ft krb5kdc[18508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091695, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:55.775876 18667 init.cc:377] Logged in from keytab as oryx/127.9.174.1@KRBTEST.COM (short username oryx)
I20260807 08:34:55.777053 18667 webserver.cc:460] Webserver started at http://127.9.174.1:45937/ using document root <none> and password file <none>
I20260807 08:34:55.777740 18667 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:55.777806 18667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:55.778054 18667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:55.780686 18667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "f41ee8b27a214c348b0e0b86a2d74f54"
format_stamp: "Formatted at 2026-08-07 08:34:55 on dist-test-slave-h3ft"
server_key: "1e4539a6997bfdb0962e9c2fed69c300"
server_key_iv: "ae345df0c041121757da445da0828702"
server_key_version: "encryptionkey@0"
I20260807 08:34:55.781316 18667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "f41ee8b27a214c348b0e0b86a2d74f54"
format_stamp: "Formatted at 2026-08-07 08:34:55 on dist-test-slave-h3ft"
server_key: "1e4539a6997bfdb0962e9c2fed69c300"
server_key_iv: "ae345df0c041121757da445da0828702"
server_key_version: "encryptionkey@0"
I20260807 08:34:55.786327 18667 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:34:55.789705 18684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:55.791510 18667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:34:55.791700 18667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "f41ee8b27a214c348b0e0b86a2d74f54"
format_stamp: "Formatted at 2026-08-07 08:34:55 on dist-test-slave-h3ft"
server_key: "1e4539a6997bfdb0962e9c2fed69c300"
server_key_iv: "ae345df0c041121757da445da0828702"
server_key_version: "encryptionkey@0"
I20260807 08:34:55.791846 18667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:55.823642 18667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:55.827832 18667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:55.828111 18667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:55.828922 18667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:55.830044 18667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:55.830121 18667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:55.830196 18667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:55.830245 18667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:55.842772 18667 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:41967
I20260807 08:34:55.842796 18844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:41967 every 8 connection(s)
I20260807 08:34:55.843999 18667 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:34:55.846606 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 18667
I20260807 08:34:55.846761 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:34:55.847186 9912 external_mini_cluster.cc:1428] Setting key 346f138cb351d79abc04b605c743e92a
Aug 07 08:34:55 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091695, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM
I20260807 08:34:55.863648 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:55.846445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:56717 (local address 127.9.174.62:42359)
0807 08:34:55.846608 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:55.846612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:55.848643 (+ 2031us) server_negotiation.cc:407] Connection header received
0807 08:34:55.849964 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:55.849971 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:55.850055 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:55.850167 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:55.852024 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:55.852796 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:55.854590 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:55.855098 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:55.855249 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:55.858703 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:55.858710 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:55.858715 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:55.858762 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:55.860878 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:55.861514 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:55.861521 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:55.861525 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:55.861650 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:55.862032 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:55.862037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:55.862039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:55.862356 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:55.862495 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:55.863151 (+ 656us) server_negotiation.cc:299] Negotiation successful
0807 08:34:55.863346 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
WARNING: no policy specified for oryx/127.9.174.2@KRBTEST.COM; defaulting to no policy
I20260807 08:34:55.864687 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:55.847157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.1:56717)
0807 08:34:55.848394 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:55.848480 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:55.849651 (+ 1171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:55.850427 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:55.850438 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:55.851040 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:55.851808 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:55.851831 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:55.852964 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:55.852968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:55.854414 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:55.854428 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:55.855307 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:55.855310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:55.855489 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:55.856295 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:55.856314 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:55.858454 (+ 2140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:55.861042 (+ 2588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:55.861048 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:55.861063 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:55.861388 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:55.861765 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:55.861767 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:55.861769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:55.861923 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:55.862513 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:55.862519 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:55.862820 (+ 301us) client_negotiation.cc:769] Sending connection context
0807 08:34:55.863060 (+ 240us) client_negotiation.cc:240] Negotiation successful
0807 08:34:55.863386 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"thread_start_us":243,"threads_started":1}
I20260807 08:34:55.866647 18845 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42359
I20260807 08:34:55.867049 18845 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.174.2@KRBTEST.COM" created.
I20260807 08:34:55.867796 18845 heartbeater.cc:511] Master 127.9.174.62:42359 requested a full tablet report, sending...
I20260807 08:34:55.870107 18567 ts_manager.cc:194] Registered new tserver with Master: f41ee8b27a214c348b0e0b86a2d74f54 (127.9.174.1:41967)
I20260807 08:34:55.871690 18567 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:56717
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.2.keytab.
Entry for principal oryx/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.2.keytab.
I20260807 08:34:55.890702 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42359
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.2.keytab
--principal=oryx/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:56.024665 18850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:56.024993 18850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:56.025067 18850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:56.029326 18850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:56.029417 18850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:56.029505 18850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:34:56.035176 18850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.2.keytab
--principal=oryx/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:56.036384 18850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:56.037494 18850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:56.048498 18861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:56.048923 18850 server_base.cc:1034] running on GCE node
W20260807 08:34:56.048532 18858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:56.048507 18859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:56.049782 18850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:56.050693 18850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:56.051991 18850 hybrid_clock.cc:648] HybridClock initialized: now 1786091696051956 us; error 51 us; skew 500 ppm
Aug 07 08:34:56 dist-test-slave-h3ft krb5kdc[18508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091696, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:56.055734 18850 init.cc:377] Logged in from keytab as oryx/127.9.174.2@KRBTEST.COM (short username oryx)
I20260807 08:34:56.056913 18850 webserver.cc:460] Webserver started at http://127.9.174.2:42405/ using document root <none> and password file <none>
I20260807 08:34:56.057672 18850 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:56.057778 18850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:56.058068 18850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:56.060643 18850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "c02c956afe4c4c4f8ff1353b2e92beb4"
format_stamp: "Formatted at 2026-08-07 08:34:56 on dist-test-slave-h3ft"
server_key: "b64869bc7eac20f2880f8a3ab8586bdc"
server_key_iv: "bd020a305b713faa9009ff678feedc51"
server_key_version: "encryptionkey@0"
I20260807 08:34:56.061312 18850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "c02c956afe4c4c4f8ff1353b2e92beb4"
format_stamp: "Formatted at 2026-08-07 08:34:56 on dist-test-slave-h3ft"
server_key: "b64869bc7eac20f2880f8a3ab8586bdc"
server_key_iv: "bd020a305b713faa9009ff678feedc51"
server_key_version: "encryptionkey@0"
I20260807 08:34:56.066183 18850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:34:56.069260 18871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:56.070900 18850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:34:56.071120 18850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "c02c956afe4c4c4f8ff1353b2e92beb4"
format_stamp: "Formatted at 2026-08-07 08:34:56 on dist-test-slave-h3ft"
server_key: "b64869bc7eac20f2880f8a3ab8586bdc"
server_key_iv: "bd020a305b713faa9009ff678feedc51"
server_key_version: "encryptionkey@0"
I20260807 08:34:56.071269 18850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:56.083873 18850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:56.087751 18850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:56.088045 18850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:56.088856 18850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:56.089960 18850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:56.090041 18850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:56.090124 18850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:56.090170 18850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:56.103466 18850 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:33389
I20260807 08:34:56.103554 19035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:33389 every 8 connection(s)
I20260807 08:34:56.104699 18850 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:34:56.111444 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 18850
I20260807 08:34:56.111629 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:34:56.112046 9912 external_mini_cluster.cc:1428] Setting key 9c62439654860ad8a225a010927241f6
Aug 07 08:34:56 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091696, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM
I20260807 08:34:56.124711 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.107472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:47751 (local address 127.9.174.62:42359)
0807 08:34:56.107733 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:56.107738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:56.109150 (+ 1412us) server_negotiation.cc:407] Connection header received
0807 08:34:56.110228 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:56.110231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:56.110316 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:56.110434 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:56.112180 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.112836 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.114769 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.115217 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.115356 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.119163 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:56.119170 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:56.119176 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:56.119219 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:56.121464 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.122353 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.122359 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.122361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.122477 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.123062 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.123067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.123069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.123389 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:56.123522 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:56.124245 (+ 723us) server_negotiation.cc:299] Negotiation successful
0807 08:34:56.124423 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
WARNING: no policy specified for oryx/127.9.174.3@KRBTEST.COM; defaulting to no policy
I20260807 08:34:56.125947 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.107960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.2:47751)
0807 08:34:56.108913 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:56.108977 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:56.109965 (+ 988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:56.110703 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:56.110711 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:56.111283 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:56.111963 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:56.111980 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.113138 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.113143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:56.114595 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:56.114613 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.115435 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.115440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:56.115621 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.116427 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:56.116447 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:56.118879 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:56.121640 (+ 2761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:56.121647 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:56.121666 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:56.122055 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:56.122681 (+ 626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:56.122684 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:56.122686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:56.122910 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:56.123533 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:56.123540 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:56.123918 (+ 378us) client_negotiation.cc:769] Sending connection context
0807 08:34:56.124134 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 08:34:56.124498 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":163,"threads_started":1}
I20260807 08:34:56.127966 19036 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42359
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.174.3@KRBTEST.COM" created.
I20260807 08:34:56.128340 19036 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:56.129201 19036 heartbeater.cc:511] Master 127.9.174.62:42359 requested a full tablet report, sending...
I20260807 08:34:56.130782 18567 ts_manager.cc:194] Registered new tserver with Master: c02c956afe4c4c4f8ff1353b2e92beb4 (127.9.174.2:33389)
I20260807 08:34:56.131644 18567 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:47751
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.3.keytab.
Entry for principal oryx/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.3.keytab.
I20260807 08:34:56.151454 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42359
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.3.keytab
--principal=oryx/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:56.278448 19043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:56.278801 19043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:56.278851 19043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:56.283150 19043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:56.283229 19043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:56.283290 19043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:34:56.289009 19043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.3.keytab
--principal=oryx/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:56.290273 19043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:56.291393 19043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:56.302587 19050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:56.302603 19055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:56.302587 19052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:56.303045 19043 server_base.cc:1034] running on GCE node
I20260807 08:34:56.303670 19043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:56.304567 19043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:56.305779 19043 hybrid_clock.cc:648] HybridClock initialized: now 1786091696305736 us; error 54 us; skew 500 ppm
Aug 07 08:34:56 dist-test-slave-h3ft krb5kdc[18508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091696, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:56.309367 19043 init.cc:377] Logged in from keytab as oryx/127.9.174.3@KRBTEST.COM (short username oryx)
I20260807 08:34:56.310583 19043 webserver.cc:460] Webserver started at http://127.9.174.3:38331/ using document root <none> and password file <none>
I20260807 08:34:56.311393 19043 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:56.311511 19043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:56.311798 19043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:56.314342 19043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "f945358cf4aa423394dbbe1bbea16832"
format_stamp: "Formatted at 2026-08-07 08:34:56 on dist-test-slave-h3ft"
server_key: "4dbe4593e9716b090d5cd42daaccacb0"
server_key_iv: "3277cf45107969859b952027f3e05e81"
server_key_version: "encryptionkey@0"
I20260807 08:34:56.315117 19043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "f945358cf4aa423394dbbe1bbea16832"
format_stamp: "Formatted at 2026-08-07 08:34:56 on dist-test-slave-h3ft"
server_key: "4dbe4593e9716b090d5cd42daaccacb0"
server_key_iv: "3277cf45107969859b952027f3e05e81"
server_key_version: "encryptionkey@0"
I20260807 08:34:56.320185 19043 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 08:34:56.323416 19067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:56.325160 19043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:34:56.325335 19043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "f945358cf4aa423394dbbe1bbea16832"
format_stamp: "Formatted at 2026-08-07 08:34:56 on dist-test-slave-h3ft"
server_key: "4dbe4593e9716b090d5cd42daaccacb0"
server_key_iv: "3277cf45107969859b952027f3e05e81"
server_key_version: "encryptionkey@0"
I20260807 08:34:56.325484 19043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:56.338577 19043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:56.342374 19043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:56.342624 19043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:56.343463 19043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:56.344533 19043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:56.344612 19043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:56.344697 19043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:56.344750 19043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:56.357836 19043 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:36867
I20260807 08:34:56.357936 19229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:36867 every 8 connection(s)
I20260807 08:34:56.359118 19043 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:34:56.361804 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 19043
I20260807 08:34:56.361979 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:34:56.362329 9912 external_mini_cluster.cc:1428] Setting key 67946fb9c35b41232776fe0780e6869a
Aug 07 08:34:56 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091696, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM
I20260807 08:34:56.376812 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.361523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:57635 (local address 127.9.174.62:42359)
0807 08:34:56.361755 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:56.361761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:56.363370 (+ 1609us) server_negotiation.cc:407] Connection header received
0807 08:34:56.364757 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:56.364760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:56.364842 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:56.364950 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:56.366688 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.367349 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.368646 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.368887 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.368985 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.372525 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:56.372529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:56.372531 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:56.372565 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:56.373966 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.374617 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.374620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.374621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.374711 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.375237 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.375240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.375241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.375475 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:56.375568 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:56.376406 (+ 838us) server_negotiation.cc:299] Negotiation successful
0807 08:34:56.376606 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 08:34:56.377996 19232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.362013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.3:57635)
0807 08:34:56.363118 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:56.363192 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:56.364404 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:56.365219 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:56.365238 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:56.365842 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:56.366510 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:56.366527 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.367480 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.367483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:56.368510 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:56.368520 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.369069 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.369072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:56.369233 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.369979 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:56.370016 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:56.372211 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:56.374120 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:56.374126 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:56.374142 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:56.374508 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:56.374855 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:56.374858 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:56.374861 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:56.375124 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:56.375679 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:56.375685 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:56.376074 (+ 389us) client_negotiation.cc:769] Sending connection context
0807 08:34:56.376330 (+ 256us) client_negotiation.cc:240] Negotiation successful
0807 08:34:56.376761 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":164,"threads_started":1}
I20260807 08:34:56.379335 19230 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42359
I20260807 08:34:56.379702 19230 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:56.380407 19230 heartbeater.cc:511] Master 127.9.174.62:42359 requested a full tablet report, sending...
I20260807 08:34:56.381726 18567 ts_manager.cc:194] Registered new tserver with Master: f945358cf4aa423394dbbe1bbea16832 (127.9.174.3:36867)
I20260807 08:34:56.382431 18567 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:57635
I20260807 08:34:56.390347 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.174.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.174.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.174.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.61.keytab.
Entry for principal oryx/127.9.174.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.61.keytab.
I20260807 08:34:56.427724 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.61:37673
--webserver_interface=127.9.174.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.61.keytab
--principal=oryx/127.9.174.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:42359,127.9.174.61:37673
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:56.571370 19238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:56.571712 19238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:56.571760 19238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:56.576370 19238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:56.576452 19238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:56.576472 19238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:56.576490 19238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:56.576507 19238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:56.582607 19238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:42359,127.9.174.61:37673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.61.keytab
--principal=oryx/127.9.174.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.61:37673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.174.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
W20260807 08:34:56.582990 19238 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.174.62:42359,127.9.174.61:37673'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 08:34:56.584278 19238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:56.585613 19238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:56.596276 19246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:56.596403 19248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:56.596276 19252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:56.596662 19238 server_base.cc:1034] running on GCE node
I20260807 08:34:56.597512 19238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:56.598963 19238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:56.600240 19238 hybrid_clock.cc:648] HybridClock initialized: now 1786091696600205 us; error 53 us; skew 500 ppm
Aug 07 08:34:56 dist-test-slave-h3ft krb5kdc[18508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091696, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:56.603966 19238 init.cc:377] Logged in from keytab as oryx/127.9.174.61@KRBTEST.COM (short username oryx)
I20260807 08:34:56.605189 19238 webserver.cc:460] Webserver started at http://127.9.174.61:39095/ using document root <none> and password file <none>
I20260807 08:34:56.605952 19238 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:56.606096 19238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:56.606371 19238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:56.609097 19238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/data/instance:
uuid: "7438c78b2f514c20babb5aae6e8bea9e"
format_stamp: "Formatted at 2026-08-07 08:34:56 on dist-test-slave-h3ft"
server_key: "88cc0d8e9753bd7f1655120b47c765a8"
server_key_iv: "e538d31f4e37822c167ddafd84159777"
server_key_version: "encryptionkey@0"
I20260807 08:34:56.609750 19238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/wal/instance:
uuid: "7438c78b2f514c20babb5aae6e8bea9e"
format_stamp: "Formatted at 2026-08-07 08:34:56 on dist-test-slave-h3ft"
server_key: "88cc0d8e9753bd7f1655120b47c765a8"
server_key_iv: "e538d31f4e37822c167ddafd84159777"
server_key_version: "encryptionkey@0"
I20260807 08:34:56.614903 19238 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:34:56.618703 19262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:56.620453 19238 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260807 08:34:56.620641 19238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/wal
uuid: "7438c78b2f514c20babb5aae6e8bea9e"
format_stamp: "Formatted at 2026-08-07 08:34:56 on dist-test-slave-h3ft"
server_key: "88cc0d8e9753bd7f1655120b47c765a8"
server_key_iv: "e538d31f4e37822c167ddafd84159777"
server_key_version: "encryptionkey@0"
I20260807 08:34:56.620787 19238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:56.635339 19238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:56.639166 19238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:56.639410 19238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:56.649143 19238 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.61:37673
I20260807 08:34:56.649510 19340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.61:37673 every 8 connection(s)
I20260807 08:34:56.650488 19238 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/data/info.pb
I20260807 08:34:56.654520 19343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:56.657980 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 19238
I20260807 08:34:56.660130 19343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:34:56 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091696, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.61@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM
Aug 07 08:34:56 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.174.61@KRBTEST.COM
I20260807 08:34:56.674424 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.662238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34742 (local address 127.9.174.62:42359)
0807 08:34:56.662394 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:56.662399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:56.662758 (+ 359us) server_negotiation.cc:407] Connection header received
0807 08:34:56.663205 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:56.663208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:56.663260 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:56.663355 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:56.664148 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.664972 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.666458 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.666794 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.666899 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.670113 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:56.670117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:56.670120 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:56.670148 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:56.672070 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.672667 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.672671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.672673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.672773 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.673159 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.673164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.673166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.673474 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:56.673602 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:56.674040 (+ 438us) server_negotiation.cc:299] Negotiation successful
0807 08:34:56.674188 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:34:56.675475 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.662120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:34742)
0807 08:34:56.662603 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:56.662664 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:56.662886 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:56.663429 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:56.663434 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:56.663808 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:56.663995 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:56.664006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.665116 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.665119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:56.666322 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:56.666332 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.667073 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.667080 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:56.667249 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.667745 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:56.667773 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:56.669844 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:56.672214 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:56.672227 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:56.672241 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:56.672533 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:56.672883 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:56.672889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:56.672894 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:56.673046 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:56.673603 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:56.673611 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:56.673781 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 08:34:56.673989 (+ 208us) client_negotiation.cc:240] Negotiation successful
0807 08:34:56.674630 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":246,"thread_start_us":145,"threads_started":1}
I20260807 08:34:56.675477 19347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.660101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43984 (local address 127.9.174.61:37673)
0807 08:34:56.660620 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:56.660640 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:56.660684 (+ 44us) server_negotiation.cc:407] Connection header received
0807 08:34:56.661468 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:56.661490 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:56.661961 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:56.662505 (+ 544us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:56.663430 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.664775 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.666085 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.666595 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.666756 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.670453 (+ 3697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:56.670459 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:56.670472 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:56.670493 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:56.672309 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.672803 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.672808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.672809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.672961 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.673259 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.673268 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.673269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.673689 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:56.673920 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:56.674149 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 08:34:56.674364 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"spinlock_wait_cycles":5888,"thread_start_us":180,"threads_started":1}
I20260807 08:34:56.678097 19343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:34:56 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091696, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.61@KRBTEST.COM for oryx/127.9.174.61@KRBTEST.COM
I20260807 08:34:56.687111 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.678677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:37673 (local address 127.0.0.1:43996)
0807 08:34:56.678832 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:56.678849 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:56.679036 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:56.679264 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:56.679267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:56.679449 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:56.679545 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:56.679551 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.680289 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.680293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:56.680822 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:56.680829 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.681292 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.681295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:56.681387 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.681899 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:56.681910 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:56.683479 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:56.685387 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:56.685390 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:56.685392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:56.685650 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:56.686047 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:56.686050 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:56.686063 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:56.686149 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:56.686565 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:56.686567 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:56.686648 (+ 81us) client_negotiation.cc:769] Sending connection context
0807 08:34:56.686735 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 08:34:56.686855 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 08:34:56.687992 19347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.678735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43996 (local address 127.9.174.61:37673)
0807 08:34:56.678832 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:56.678837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:56.678917 (+ 80us) server_negotiation.cc:407] Connection header received
0807 08:34:56.679125 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:56.679128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:56.679172 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:56.679251 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:56.679654 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.680182 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.680936 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.681175 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.681282 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.683621 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:56.683624 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:56.683627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:56.683649 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:56.685235 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.685793 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.685796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.685797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.685926 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.686250 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.686253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.686254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.686476 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:56.686561 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:56.687753 (+ 1192us) server_negotiation.cc:299] Negotiation successful
0807 08:34:56.687885 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 08:34:56.691718 19343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: Bootstrap starting.
I20260807 08:34:56.694617 19343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:56.695608 19343 log.cc:824] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:56.698099 19343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: No bootstrap required, opened a new log
I20260807 08:34:56.701596 19343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } }
I20260807 08:34:56.701825 19343 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:56.701867 19343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7438c78b2f514c20babb5aae6e8bea9e, State: Initialized, Role: FOLLOWER
I20260807 08:34:56.702306 19343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } }
I20260807 08:34:56.702865 19352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } } }
I20260807 08:34:56.703063 19352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:56.703766 19343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: configured and running, proceeding with master startup.
W20260807 08:34:56.709889 19365 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:34:56.710039 19365 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:34:56.713385 19238 master_runner.cc:428] Detected that this master 7438c78b2f514c20babb5aae6e8bea9e is joining an existing cluster
I20260807 08:34:56.713469 19238 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.174.61:37673
I20260807 08:34:56.725708 19347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.716259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44004 (local address 127.9.174.61:37673)
0807 08:34:56.716548 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:56.716553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:56.716694 (+ 141us) server_negotiation.cc:407] Connection header received
0807 08:34:56.716930 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:56.716933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:56.716986 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:56.717059 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:56.717717 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.718530 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.719785 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.720146 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.720259 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.721573 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:56.721577 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:56.721580 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:56.721628 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:56.723598 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.724212 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.724216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.724218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.724308 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.724678 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.724683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.724686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.724937 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:56.725041 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:56.725369 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 08:34:56.725580 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":205}
I20260807 08:34:56.726397 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.716173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34754 (local address 127.9.174.62:42359)
0807 08:34:56.716436 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:56.716442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:56.716691 (+ 249us) server_negotiation.cc:407] Connection header received
0807 08:34:56.716842 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:56.716847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:56.716914 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:56.717030 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:56.717676 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.718376 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.719554 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.719835 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.719947 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.721678 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:56.721683 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:56.721688 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:56.721714 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:56.723686 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.724289 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.724292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.724294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.724389 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.724719 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.724724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.724728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.724937 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:56.725021 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:56.726083 (+ 1062us) server_negotiation.cc:299] Negotiation successful
0807 08:34:56.726222 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260807 08:34:56.726938 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.716122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:34754)
0807 08:34:56.716581 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:56.716597 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:56.716733 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:56.717047 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:56.717050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:56.717360 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:56.717518 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:56.717525 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.718547 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.718550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:56.719427 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:56.719437 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.719993 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.719997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:56.720172 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.720838 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:56.720855 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:56.721479 (+ 624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:56.723796 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:56.723800 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:56.723802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:56.724188 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:56.724506 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:56.724511 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:56.724517 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:56.724592 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:56.725093 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:56.725096 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:56.725864 (+ 768us) client_negotiation.cc:769] Sending connection context
0807 08:34:56.726622 (+ 758us) client_negotiation.cc:240] Negotiation successful
0807 08:34:56.726768 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":721,"thread_start_us":120,"threads_started":1}
I20260807 08:34:56.725737 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.716185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:37673 (local address 127.0.0.1:44004)
0807 08:34:56.716583 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:56.716598 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:56.716759 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:56.717192 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:56.717195 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:56.717461 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:56.717595 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:56.717603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.718669 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.718672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:56.719661 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:56.719672 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.720266 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.720270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:56.720384 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.721011 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:56.721032 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:56.721423 (+ 391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:56.723728 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:56.723732 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:56.723736 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:56.724081 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:56.724460 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:56.724464 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:56.724466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:56.724570 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:56.725044 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:56.725047 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:56.725187 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 08:34:56.725298 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 08:34:56.725466 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":42,"thread_start_us":128,"threads_started":1}
Aug 07 08:34:56 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091695, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.62@KRBTEST.COM for oryx/127.9.174.61@KRBTEST.COM
I20260807 08:34:56.747125 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.734164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:37673 (local address 127.0.0.1:44016)
0807 08:34:56.735183 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:56.735247 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:56.735401 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:56.735842 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:56.735850 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:56.736379 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:56.736542 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:56.736550 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.737495 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.737499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:56.738556 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:56.738566 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.739168 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.739172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:56.739305 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.740023 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:56.740040 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:56.742052 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:56.744701 (+ 2649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:56.744712 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:56.744721 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:56.745098 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:56.745492 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:56.745495 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:56.745497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:56.745601 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:56.746259 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:56.746268 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:56.746428 (+ 160us) client_negotiation.cc:769] Sending connection context
0807 08:34:56.746608 (+ 180us) client_negotiation.cc:240] Negotiation successful
0807 08:34:56.746856 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"thread_start_us":124,"threads_started":1}
I20260807 08:34:56.747694 19347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.734260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44016 (local address 127.9.174.61:37673)
0807 08:34:56.734435 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:56.734450 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:56.735350 (+ 900us) server_negotiation.cc:407] Connection header received
0807 08:34:56.735581 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:56.735587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:56.735671 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:56.735792 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:56.736687 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.737357 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.738705 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.738964 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.739173 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.742231 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:56.742239 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:56.742246 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:56.742287 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:56.744561 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.745230 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.745238 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.745243 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.745366 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.745725 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.745732 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.745737 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.746107 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:56.746258 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:56.746730 (+ 472us) server_negotiation.cc:299] Negotiation successful
0807 08:34:56.746926 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 08:34:56.749732 18567 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.174.61:37673: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e"
member_type: NON_VOTER
last_known_addr {
host: "127.9.174.61"
port: 37673
}
attrs {
promote: true
}
}
dest_uuid: "5c7b45d28ba243aa8679d3071c58d4c9"
cas_config_opid_index: -1
WARNING: no policy specified for oryx/127.9.174.60@KRBTEST.COM; defaulting to no policy
I20260807 08:34:56.750682 18567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.174.60@KRBTEST.COM" created.
I20260807 08:34:56.753389 19383 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7438c78b2f514c20babb5aae6e8bea9e (127.9.174.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } } }
I20260807 08:34:56.754626 19302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:56.756071 19383 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.174.61:37673
I20260807 08:34:56.756392 19302 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 FOLLOWER]: Refusing update from remote peer 5c7b45d28ba243aa8679d3071c58d4c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 08:34:56.756745 19384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } } }
I20260807 08:34:56.756925 19384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:56.757033 19238 master.cc:537] Master@127.9.174.61:37673 shutting down...
I20260807 08:34:56.758080 19384 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 08:34:56.760432 18553 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 08:34:56.761435 18553 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 -> Peer 7438c78b2f514c20babb5aae6e8bea9e (127.9.174.61:37673): Couldn't send request to peer 7438c78b2f514c20babb5aae6e8bea9e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 08:34:56.763572 19238 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:34:56.764008 19238 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:56.764189 19238 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: stopping tablet replica
I20260807 08:34:56.772404 19238 master.cc:552] Master@127.9.174.61:37673 shutdown complete.
I20260807 08:34:56.772482 19238 master_runner.cc:305] Clearing existing system tablet
I20260807 08:34:56.772915 19238 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:56.773003 19238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:56.774973 19238 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 08:34:56.776297 19391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:56.776607 19238 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:56.776736 19238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:34:56.776763 19238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/wal
uuid: "7438c78b2f514c20babb5aae6e8bea9e"
format_stamp: "Formatted at 2026-08-07 08:34:56 on dist-test-slave-h3ft"
server_key: "88cc0d8e9753bd7f1655120b47c765a8"
server_key_iv: "e538d31f4e37822c167ddafd84159777"
server_key_version: "encryptionkey@0"
I20260807 08:34:56.779510 19238 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.174.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.60.keytab.
Entry for principal oryx/127.9.174.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.60.keytab.
I20260807 08:34:56.783519 19238 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 08:34:56.783656 19238 log.cc:1196] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 08:34:56.783982 19238 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: Deleting consensus metadata
I20260807 08:34:56.784125 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.60:45863
--webserver_interface=127.9.174.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.60.keytab
--principal=oryx/127.9.174.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:42359,127.9.174.61:37673,127.9.174.60:45863
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:34:56.784191 19238 master_runner.cc:315] Copying system tablet from 127.9.174.62:42359
I20260807 08:34:56.785931 19238 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: tablet copy: Beginning tablet copy session from remote peer at address 127.9.174.62:42359
I20260807 08:34:56.796757 19400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.786785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:34766)
0807 08:34:56.787336 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:56.787371 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:56.787546 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:56.787906 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:56.787911 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:56.788226 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:56.788441 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:56.788454 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.789742 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.789746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:56.790478 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:56.790490 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.791084 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.791089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:56.791206 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.792426 (+ 1220us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:56.792439 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:56.792846 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:56.794792 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:56.794797 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:56.794799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:56.795115 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:56.795441 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:56.795444 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:56.795446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:56.795545 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:56.795997 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:56.796000 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:56.796175 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 08:34:56.796310 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 08:34:56.796515 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":301,"threads_started":1}
I20260807 08:34:56.796742 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:56.786980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34766 (local address 127.9.174.62:42359)
0807 08:34:56.787242 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:56.787249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:56.787484 (+ 235us) server_negotiation.cc:407] Connection header received
0807 08:34:56.787672 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:56.787676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:56.787744 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:56.787860 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:56.788606 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.789591 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.790619 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:56.790878 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:56.791005 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:56.793010 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:56.793015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:56.793018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:56.793054 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:56.794654 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.795234 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.795238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.795240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.795333 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:56.795658 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:56.795661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:56.795663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:56.795891 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:56.796007 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:56.796389 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 08:34:56.796524 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260807 08:34:56.797987 18603 tablet_copy_service.cc:140] P 5c7b45d28ba243aa8679d3071c58d4c9: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7438c78b2f514c20babb5aae6e8bea9e ({username='oryx', principal='oryx/127.9.174.61@KRBTEST.COM'} at 127.0.0.1:34766)
I20260807 08:34:56.798225 18603 tablet_copy_service.cc:161] P 5c7b45d28ba243aa8679d3071c58d4c9: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7438c78b2f514c20babb5aae6e8bea9e at {username='oryx', principal='oryx/127.9.174.61@KRBTEST.COM'} at 127.0.0.1:34766: session id = 7438c78b2f514c20babb5aae6e8bea9e-00000000000000000000000000000000
I20260807 08:34:56.800328 18603 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 08:34:56.802459 19238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 08:34:56.805693 19238 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: tablet copy: Starting download of 0 data blocks...
I20260807 08:34:56.806005 19238 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: tablet copy: Starting download of 1 WAL segments...
I20260807 08:34:56.808673 19238 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 08:34:56.810881 18603 tablet_copy_service.cc:342] P 5c7b45d28ba243aa8679d3071c58d4c9: Request end of tablet copy session 7438c78b2f514c20babb5aae6e8bea9e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.174.61@KRBTEST.COM'} at 127.0.0.1:34766
I20260807 08:34:56.811173 18603 tablet_copy_service.cc:433] P 5c7b45d28ba243aa8679d3071c58d4c9: ending tablet copy session 7438c78b2f514c20babb5aae6e8bea9e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7438c78b2f514c20babb5aae6e8bea9e
I20260807 08:34:56.821305 19238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:56.823412 19404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:56.823544 19401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:56.823763 19402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:56.824345 19238 server_base.cc:1034] running on GCE node
I20260807 08:34:56.824544 19238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:56.824982 19238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:56.826159 19238 hybrid_clock.cc:648] HybridClock initialized: now 1786091696826144 us; error 34 us; skew 500 ppm
Aug 07 08:34:56 dist-test-slave-h3ft krb5kdc[18508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091696, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:56.828042 19238 init.cc:377] Logged in from keytab as oryx/127.9.174.61@KRBTEST.COM (short username oryx)
I20260807 08:34:56.828627 19238 webserver.cc:460] Webserver started at http://127.9.174.61:44471/ using document root <none> and password file <none>
I20260807 08:34:56.828868 19238 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:56.828928 19238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:56.830580 19238 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:34:56.831857 19414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:56.832158 19238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:34:56.832262 19238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/wal
uuid: "7438c78b2f514c20babb5aae6e8bea9e"
format_stamp: "Formatted at 2026-08-07 08:34:56 on dist-test-slave-h3ft"
server_key: "88cc0d8e9753bd7f1655120b47c765a8"
server_key_iv: "e538d31f4e37822c167ddafd84159777"
server_key_version: "encryptionkey@0"
I20260807 08:34:56.832427 19238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:56.862270 19238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:56.864789 19238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:56.871098 19238 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.61:37673
I20260807 08:34:56.871115 19487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.61:37673 every 8 connection(s)
I20260807 08:34:56.871841 19238 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-1/data/info.pb
I20260807 08:34:56.873750 19488 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:34:56.874846 19488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: Bootstrap starting.
I20260807 08:34:56.875536 18845 heartbeater.cc:503] Master 127.9.174.62:42359 was elected leader, sending a full tablet report...
I20260807 08:34:56.886127 19488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:34:56.886603 19488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: Bootstrap complete.
I20260807 08:34:56.887436 19488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } }
I20260807 08:34:56.887603 19488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7438c78b2f514c20babb5aae6e8bea9e, State: Initialized, Role: LEARNER
I20260807 08:34:56.887739 19488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } }
I20260807 08:34:56.888219 19491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } } }
I20260807 08:34:56.888428 19491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: This master's current role is: LEARNER
I20260807 08:34:56.888430 19488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:56.892369 19510 catalog_manager.cc:1259] Loaded cluster ID: 52ca67c8727643308ba3e9a056ffe10c
I20260807 08:34:56.892458 19510 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: loading cluster ID for follower catalog manager: success
I20260807 08:34:56.894245 19510 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: acquiring CA information for follower catalog manager: success
I20260807 08:34:56.895823 19510 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 08:34:56.976814 19399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:56.977217 19399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:56.977289 19399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:56.981884 19399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:56.981989 19399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:56.982012 19399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:56.982031 19399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:56.982048 19399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:56.988641 19399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:42359,127.9.174.61:37673,127.9.174.60:45863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.60.keytab
--principal=oryx/127.9.174.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.60:45863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.174.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:56.990360 19399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:56.992128 19399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:34:57.003067 19399 server_base.cc:1034] running on GCE node
W20260807 08:34:57.002856 19519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:57.003110 19516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:57.002856 19517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:57.004130 19399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:57.005544 19399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:57.006737 19399 hybrid_clock.cc:648] HybridClock initialized: now 1786091697006707 us; error 42 us; skew 500 ppm
Aug 07 08:34:57 dist-test-slave-h3ft krb5kdc[18508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:57.010480 19399 init.cc:377] Logged in from keytab as oryx/127.9.174.60@KRBTEST.COM (short username oryx)
I20260807 08:34:57.011718 19399 webserver.cc:460] Webserver started at http://127.9.174.60:42873/ using document root <none> and password file <none>
I20260807 08:34:57.012431 19399 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:57.012502 19399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:57.012746 19399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:57.015628 19399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/data/instance:
uuid: "ea61347409eb4bfc818fa544931f8576"
format_stamp: "Formatted at 2026-08-07 08:34:57 on dist-test-slave-h3ft"
server_key: "c413e6972d943aac66e9e1c89201b78b"
server_key_iv: "121de507c3481553db52fa742523ca8b"
server_key_version: "encryptionkey@0"
I20260807 08:34:57.016470 19399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/wal/instance:
uuid: "ea61347409eb4bfc818fa544931f8576"
format_stamp: "Formatted at 2026-08-07 08:34:57 on dist-test-slave-h3ft"
server_key: "c413e6972d943aac66e9e1c89201b78b"
server_key_iv: "121de507c3481553db52fa742523ca8b"
server_key_version: "encryptionkey@0"
I20260807 08:34:57.022099 19399 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:34:57.025434 19530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:57.027216 19399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 08:34:57.027390 19399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/wal
uuid: "ea61347409eb4bfc818fa544931f8576"
format_stamp: "Formatted at 2026-08-07 08:34:57 on dist-test-slave-h3ft"
server_key: "c413e6972d943aac66e9e1c89201b78b"
server_key_iv: "121de507c3481553db52fa742523ca8b"
server_key_version: "encryptionkey@0"
I20260807 08:34:57.027541 19399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:57.042510 19399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:57.046553 19399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:57.046803 19399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:57.056625 19399 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.60:45863
I20260807 08:34:57.057039 19607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.60:45863 every 8 connection(s)
I20260807 08:34:57.057942 19399 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/data/info.pb
I20260807 08:34:57.061743 19609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:57.064990 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 19399
I20260807 08:34:57.067417 19609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:34:57 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.60@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM
Aug 07 08:34:57 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.174.60@KRBTEST.COM
I20260807 08:34:57.082667 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.069809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34778 (local address 127.9.174.62:42359)
0807 08:34:57.069978 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.069983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.070594 (+ 611us) server_negotiation.cc:407] Connection header received
0807 08:34:57.070737 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.070740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.070805 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.070916 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:57.071856 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.072491 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.073684 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.074021 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.074122 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.077484 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:57.077489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:57.077492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:57.077530 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:57.079430 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.080207 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.080210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.080212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.080340 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.080849 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.080852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.080854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.081237 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:57.081323 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.082263 (+ 940us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.082440 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 08:34:57.083945 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.067225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42212 (local address 127.9.174.60:45863)
0807 08:34:57.067892 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.067942 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.068005 (+ 63us) server_negotiation.cc:407] Connection header received
0807 08:34:57.068784 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.068805 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.069225 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.069618 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:57.070641 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.072040 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.073167 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.073780 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.074028 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.077842 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:57.077849 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:57.077863 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:57.077885 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:57.080195 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.080735 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.080745 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.080750 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.080850 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.081191 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.081194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.081195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.081922 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:57.082157 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.082413 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.082672 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":481,"spinlock_wait_cycles":2432,"thread_start_us":252,"threads_started":1}
I20260807 08:34:57.083946 19612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.069656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:34778)
0807 08:34:57.070434 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:57.070498 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:57.070635 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:57.071067 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:57.071075 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:57.071506 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:57.071699 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.071710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.072650 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.072654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.073557 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.073567 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.074209 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.074214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.074360 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.074942 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:57.074967 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:57.077228 (+ 2261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:57.079564 (+ 2336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.079575 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.079589 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.080078 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.080482 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.080485 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.080487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.080699 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.081348 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:57.081356 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:57.081922 (+ 566us) client_negotiation.cc:769] Sending connection context
0807 08:34:57.082189 (+ 267us) client_negotiation.cc:240] Negotiation successful
0807 08:34:57.083193 (+ 1004us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":632,"spinlock_wait_cycles":14464,"thread_start_us":167,"threads_started":1}
I20260807 08:34:57.085743 19609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:34:57 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.60@KRBTEST.COM for oryx/127.9.174.61@KRBTEST.COM
I20260807 08:34:57.096359 19612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.086487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:37673 (local address 127.0.0.1:44028)
0807 08:34:57.086658 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:57.086675 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:57.086778 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:57.087413 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:57.087416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:57.087633 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:57.087765 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.087771 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.088775 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.088781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.089368 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.089375 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.089986 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.089990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.090119 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.090730 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:57.090760 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:57.092425 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:57.094418 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.094421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.094423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.094661 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.095115 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.095121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.095123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.095227 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.095715 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:57.095718 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:57.095829 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 08:34:57.095997 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 08:34:57.096190 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 08:34:57.096544 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.086653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44028 (local address 127.9.174.61:37673)
0807 08:34:57.087106 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.087110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.087124 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:34:57.087195 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.087201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.087270 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.087395 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:57.087899 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.088654 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.089501 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.089855 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.089971 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.092603 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:57.092606 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:57.092609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:57.092637 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:57.094290 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.094786 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.094789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.094790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.094931 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.095342 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.095345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.095346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.095608 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:57.095697 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.096133 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.096313 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":138,"threads_started":1}
I20260807 08:34:57.097496 19609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:34:57 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.60@KRBTEST.COM for oryx/127.9.174.60@KRBTEST.COM
I20260807 08:34:57.107921 19612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.098145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45863 (local address 127.0.0.1:42216)
0807 08:34:57.098335 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:57.098350 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:57.098501 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:57.098758 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:57.098761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:57.098955 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:57.099096 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.099103 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.100416 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.100422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.100991 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.100999 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.101457 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.101469 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.101572 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.102139 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:57.102150 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:57.103828 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:57.105986 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.105989 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.105991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.106254 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.106631 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.106635 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.106636 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.106759 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.107309 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:57.107312 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:57.107444 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 08:34:57.107554 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 08:34:57.107701 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":5}
I20260807 08:34:57.107944 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.098247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42216 (local address 127.9.174.60:45863)
0807 08:34:57.098379 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.098384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.098439 (+ 55us) server_negotiation.cc:407] Connection header received
0807 08:34:57.098615 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.098619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.098664 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.098747 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:57.099216 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.100300 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.101108 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.101344 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.101456 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.103992 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:57.103997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:57.104000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:57.104026 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:57.105835 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.106382 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.106386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.106388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.106487 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.106867 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.106870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.106872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.107182 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:57.107285 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.107629 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.107763 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":62}
I20260807 08:34:57.112073 19609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: Bootstrap starting.
I20260807 08:34:57.115466 19609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:57.117071 19609 log.cc:824] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:57.120304 19609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: No bootstrap required, opened a new log
I20260807 08:34:57.125520 19609 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } }
I20260807 08:34:57.125871 19609 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:57.125923 19609 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea61347409eb4bfc818fa544931f8576, State: Initialized, Role: FOLLOWER
I20260807 08:34:57.126507 19609 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } }
I20260807 08:34:57.127457 19615 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } } }
I20260807 08:34:57.127621 19615 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:57.128414 19609 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:57.135134 19036 heartbeater.cc:503] Master 127.9.174.62:42359 was elected leader, sending a full tablet report...
W20260807 08:34:57.136119 19632 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:34:57.136219 19632 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:34:57.138672 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 18667
I20260807 08:34:57.147387 19399 master_runner.cc:428] Detected that this master ea61347409eb4bfc818fa544931f8576 is joining an existing cluster
I20260807 08:34:57.147507 19399 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.174.60:45863
I20260807 08:34:57.158013 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:41967
--local_ip_for_outbound_sockets=127.9.174.1
--tserver_master_addrs=127.9.174.62:42359
--webserver_port=45937
--webserver_interface=127.9.174.1
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.1.keytab
--principal=oryx/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.174.62:42359,127.9.174.61:37673
--tserver_master_addrs=127.9.174.62:42359,127.9.174.61:37673,127.9.174.60:45863 with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:34:57.163182 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.151609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34782 (local address 127.9.174.62:42359)
0807 08:34:57.151749 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.151756 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.152273 (+ 517us) server_negotiation.cc:407] Connection header received
0807 08:34:57.152518 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.152522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.152605 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.152674 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:57.153984 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.154885 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.156585 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.156926 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.157006 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.158699 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:57.158707 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:57.158713 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:57.158750 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:57.160740 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.161438 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.161444 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.161449 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.161549 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.161900 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.161918 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.161923 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.162193 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:57.162286 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.162680 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.162827 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:34:57.163887 19643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.150976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:34782)
0807 08:34:57.152165 (+ 1189us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:57.152182 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:57.152378 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:57.152938 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:57.152941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:57.153381 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:57.153806 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.153815 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.155088 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.155095 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.156468 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.156476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.157168 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.157171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.157328 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.157965 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:57.157980 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:57.158528 (+ 548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:57.160887 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.160894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.160897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.161309 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.161678 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.161683 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.161687 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.161793 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.162360 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:57.162363 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:57.162473 (+ 110us) client_negotiation.cc:769] Sending connection context
0807 08:34:57.163487 (+ 1014us) client_negotiation.cc:240] Negotiation successful
0807 08:34:57.163666 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1039,"mutex_wait_us":30,"thread_start_us":131,"threads_started":1}
I20260807 08:34:57.169269 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.152490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42218 (local address 127.9.174.60:45863)
0807 08:34:57.152607 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.152612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.157290 (+ 4678us) server_negotiation.cc:407] Connection header received
0807 08:34:57.160579 (+ 3289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.160582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.160623 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.160669 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:57.161410 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.162207 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.163739 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.164062 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.164152 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.165989 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:57.165993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:57.165996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:57.166016 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:57.167597 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.167978 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.167984 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.167986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.168076 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.168365 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.168369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.168371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.168596 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:57.168710 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.168965 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.169120 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":41}
I20260807 08:34:57.169286 19642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.152172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45863 (local address 127.0.0.1:42218)
0807 08:34:57.157189 (+ 5017us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:57.157208 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:57.160450 (+ 3242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:57.160809 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:57.160812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:57.161130 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:57.161288 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.161296 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.162306 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.162312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.163615 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.163627 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.164625 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.164629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.164735 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.165373 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:57.165388 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:57.165851 (+ 463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:57.167693 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.167695 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.167696 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.167860 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.168172 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.168177 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.168178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.168267 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.168697 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:57.168700 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:57.168786 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 08:34:57.168901 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 08:34:57.169061 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4928,"thread_start_us":106,"threads_started":1}
I20260807 08:34:57.176287 19644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.150976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:37673 (local address 127.0.0.1:44036)
0807 08:34:57.152811 (+ 1835us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:57.152825 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:57.153001 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:57.153608 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:57.153611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:57.153865 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:57.154025 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.154033 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.155147 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.155149 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.156235 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.156246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.157062 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.157065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.157204 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.171690 (+ 14486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:57.171709 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:57.172168 (+ 459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:57.174321 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.174328 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.174332 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.174596 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.174970 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.174974 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.174976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.175106 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.175727 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:57.175730 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:57.175825 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 08:34:57.175924 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 08:34:57.176075 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1746,"mutex_wait_us":729,"thread_start_us":124,"threads_started":1}
I20260807 08:34:57.177199 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.151253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44036 (local address 127.9.174.61:37673)
0807 08:34:57.151424 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.151430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.152928 (+ 1498us) server_negotiation.cc:407] Connection header received
0807 08:34:57.153147 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.153151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.153258 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.153323 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:57.154168 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.154988 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.156380 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.156761 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.156841 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.172415 (+ 15574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:57.172421 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:57.172424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:57.172468 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:57.174191 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.174729 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.174734 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.174736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.174844 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.175305 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.175309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.175311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.175591 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:57.175687 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.176850 (+ 1163us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.177025 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 08:34:57 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091695, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.62@KRBTEST.COM for oryx/127.9.174.60@KRBTEST.COM
I20260807 08:34:57.183941 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.174419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42222 (local address 127.9.174.60:45863)
0807 08:34:57.174564 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.174568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.174581 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:34:57.174663 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.174667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.174727 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.174798 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:57.175596 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.176342 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.178111 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.178396 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.178508 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.180661 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:57.180664 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:57.180666 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:57.180687 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:57.182152 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.182608 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.182613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.182615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.182697 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.182992 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.182997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.183040 (+ 43us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.183239 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:57.183355 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.183632 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.183753 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 08:34:57.183941 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.173512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45863 (local address 127.0.0.1:42222)
0807 08:34:57.173674 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:57.173691 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:57.173783 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:57.174982 (+ 1199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:57.174985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:57.175254 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:57.175434 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.175444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.176658 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.176661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.177489 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.177500 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.178510 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.178514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.178616 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.179138 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:57.179150 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:57.180511 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:57.182271 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.182276 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.182279 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.182506 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.182796 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.182799 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.182801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.182900 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.183343 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:57.183347 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:57.183467 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 08:34:57.183580 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 08:34:57.183766 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 08:34:57.185509 18567 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.174.60:45863: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ea61347409eb4bfc818fa544931f8576"
member_type: NON_VOTER
last_known_addr {
host: "127.9.174.60"
port: 45863
}
attrs {
promote: true
}
}
dest_uuid: "5c7b45d28ba243aa8679d3071c58d4c9"
cas_config_opid_index: 5
I20260807 08:34:57.186301 18567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } }
I20260807 08:34:57.187777 19383 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ea61347409eb4bfc818fa544931f8576 (127.9.174.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } } }
I20260807 08:34:57.189685 19383 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.174.60:45863
I20260807 08:34:57.189937 19383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } } }
I20260807 08:34:57.190054 19383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:57.191164 19569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:57.192668 19569 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 FOLLOWER]: Refusing update from remote peer 5c7b45d28ba243aa8679d3071c58d4c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 08:34:57.193642 19384 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:57.194749 19399 master.cc:537] Master@127.9.174.60:45863 shutting down...
I20260807 08:34:57.199846 19569 raft_consensus.cc:858] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } }
W20260807 08:34:57.199971 19569 raft_consensus.cc:1565] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } }
I20260807 08:34:57.201709 19569 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7438c78b2f514c20babb5aae6e8bea9e (127.9.174.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } } }
I20260807 08:34:57.203032 19647 mvcc.cc:205] Tried to move back new op lower bound from 7315831584981442560 to 7315831584832819200. Current Snapshot: MvccSnapshot[applied={T|T < 7315831584872292352}]
I20260807 08:34:57.208739 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.188030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:37673 (local address 127.0.0.1:44048)
0807 08:34:57.203204 (+ 15174us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:57.203232 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:57.203503 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:57.204055 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:57.204058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:57.204078 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:57.204219 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.204226 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.205373 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.205376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.206828 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.206841 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.208169 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.208173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.208282 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.208345 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 08:34:57.208446 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 08:34:57.208554 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15016}
I20260807 08:34:57.208779 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.188327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44048 (local address 127.9.174.61:37673)
0807 08:34:57.188507 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.188513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.203362 (+ 14849us) server_negotiation.cc:407] Connection header received
0807 08:34:57.203674 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.203678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.203768 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.203842 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:57.204363 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.205214 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.207047 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.207783 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.207876 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.208053 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.208505 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.208625 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 08:34:57.209554 19456 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 LEARNER]: Refusing update from remote peer 5c7b45d28ba243aa8679d3071c58d4c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 08:34:57.210161 19384 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 08:34:57.212327 19654 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } } }
I20260807 08:34:57.212520 19654 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 08:34:57.213255 19456 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ea61347409eb4bfc818fa544931f8576 (127.9.174.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } } }
I20260807 08:34:57.214779 19615 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c7b45d28ba243aa8679d3071c58d4c9. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } } }
I20260807 08:34:57.214938 19615 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 08:34:57.214948 19399 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260807 08:34:57.215395 19399 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260807 08:34:57.215728 19384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } } }
I20260807 08:34:57.215894 19384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: This master's current role is: LEADER
W20260807 08:34:57.217419 18552 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 -> Peer ea61347409eb4bfc818fa544931f8576 (127.9.174.60:45863): Couldn't send request to peer ea61347409eb4bfc818fa544931f8576. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 08:34:57.217691 19650 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } } }
I20260807 08:34:57.217808 19650 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 08:34:57.217832 19491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c7b45d28ba243aa8679d3071c58d4c9. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } } }
I20260807 08:34:57.217993 19491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: This master's current role is: LEARNER
I20260807 08:34:57.218158 19399 tablet_replica.cc:330] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: stopping tablet replica
I20260807 08:34:57.218235 19658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: NON_VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: true } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } } }
I20260807 08:34:57.218365 19658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: This master's current role is: LEARNER
I20260807 08:34:57.219789 19384 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9: attempting to promote NON_VOTER 7438c78b2f514c20babb5aae6e8bea9e to VOTER
I20260807 08:34:57.220352 19384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } }
W20260807 08:34:57.222959 18552 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 -> Peer ea61347409eb4bfc818fa544931f8576 (127.9.174.60:45863): Couldn't send request to peer ea61347409eb4bfc818fa544931f8576. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260807 08:34:57.223843 19456 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 LEARNER]: Refusing update from remote peer 5c7b45d28ba243aa8679d3071c58d4c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 08:34:57.224423 19659 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 08:34:57.226609 19399 master.cc:552] Master@127.9.174.60:45863 shutdown complete.
I20260807 08:34:57.226678 19399 master_runner.cc:305] Clearing existing system tablet
I20260807 08:34:57.226989 19399 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:57.227090 19399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:57.228569 19399 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:34:57.228343 19384 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7438c78b2f514c20babb5aae6e8bea9e (127.9.174.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } } }
I20260807 08:34:57.229552 19661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:57.229750 19399 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:57.229861 19399 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:34:57.229892 19399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/wal
uuid: "ea61347409eb4bfc818fa544931f8576"
format_stamp: "Formatted at 2026-08-07 08:34:57 on dist-test-slave-h3ft"
server_key: "c413e6972d943aac66e9e1c89201b78b"
server_key_iv: "121de507c3481553db52fa742523ca8b"
server_key_version: "encryptionkey@0"
I20260807 08:34:57.231835 19383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } } }
I20260807 08:34:57.231990 19383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:57.232131 19383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } } }
I20260807 08:34:57.232198 19383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:57.232187 19399 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:57.233712 19456 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7438c78b2f514c20babb5aae6e8bea9e (127.9.174.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } } }
I20260807 08:34:57.234985 19399 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 08:34:57.235143 19399 log.cc:1196] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 08:34:57.235474 19399 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: Deleting consensus metadata
I20260807 08:34:57.235651 19399 master_runner.cc:315] Copying system tablet from 127.9.174.62:42359
I20260807 08:34:57.237035 19399 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: tablet copy: Beginning tablet copy session from remote peer at address 127.9.174.62:42359
I20260807 08:34:57.237099 19658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } } }
I20260807 08:34:57.237746 19658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:57.248268 19671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.238122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:34788)
0807 08:34:57.238397 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:57.238416 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:57.238561 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:57.238890 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:57.238894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:57.239218 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:57.239421 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.239437 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.240626 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.240632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.241652 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.241665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.242282 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.242286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.242383 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.243141 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:57.243157 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:57.243688 (+ 531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:57.245881 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.245885 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.245888 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.246181 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.246820 (+ 639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.246824 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.246826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.246938 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.247537 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:57.247541 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:57.247712 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 08:34:57.247815 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 08:34:57.248013 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":93,"threads_started":1}
I20260807 08:34:57.248287 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.237961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34788 (local address 127.9.174.62:42359)
0807 08:34:57.238139 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.238145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.238495 (+ 350us) server_negotiation.cc:407] Connection header received
0807 08:34:57.238683 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.238687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.238750 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.238847 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:57.239582 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.240473 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.241790 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.242148 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.242258 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.243855 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:57.243859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:57.243862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:57.243891 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:57.245750 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.246296 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.246300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.246302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.246700 (+ 398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.247139 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.247142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.247144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.247414 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:57.247520 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.247952 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.248108 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 08:34:57.249684 18603 tablet_copy_service.cc:140] P 5c7b45d28ba243aa8679d3071c58d4c9: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ea61347409eb4bfc818fa544931f8576 ({username='oryx', principal='oryx/127.9.174.60@KRBTEST.COM'} at 127.0.0.1:34788)
I20260807 08:34:57.249822 18603 tablet_copy_service.cc:161] P 5c7b45d28ba243aa8679d3071c58d4c9: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ea61347409eb4bfc818fa544931f8576 at {username='oryx', principal='oryx/127.9.174.60@KRBTEST.COM'} at 127.0.0.1:34788: session id = ea61347409eb4bfc818fa544931f8576-00000000000000000000000000000000
I20260807 08:34:57.251235 18603 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 08:34:57.253047 19399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 08:34:57.256331 19399 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: tablet copy: Starting download of 0 data blocks...
I20260807 08:34:57.256616 19399 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: tablet copy: Starting download of 1 WAL segments...
I20260807 08:34:57.258692 19399 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 08:34:57.261272 18603 tablet_copy_service.cc:342] P 5c7b45d28ba243aa8679d3071c58d4c9: Request end of tablet copy session ea61347409eb4bfc818fa544931f8576-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.174.60@KRBTEST.COM'} at 127.0.0.1:34788
I20260807 08:34:57.261410 18603 tablet_copy_service.cc:433] P 5c7b45d28ba243aa8679d3071c58d4c9: ending tablet copy session ea61347409eb4bfc818fa544931f8576-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ea61347409eb4bfc818fa544931f8576
I20260807 08:34:57.271775 19399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:57.274099 19672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:57.274115 19676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:57.275189 19399 server_base.cc:1034] running on GCE node
W20260807 08:34:57.275281 19673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:57.275525 19399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:57.275951 19399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:57.277097 19399 hybrid_clock.cc:648] HybridClock initialized: now 1786091697277071 us; error 44 us; skew 500 ppm
Aug 07 08:34:57 dist-test-slave-h3ft krb5kdc[18508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:57.279213 19399 init.cc:377] Logged in from keytab as oryx/127.9.174.60@KRBTEST.COM (short username oryx)
I20260807 08:34:57.279874 19399 webserver.cc:460] Webserver started at http://127.9.174.60:46045/ using document root <none> and password file <none>
I20260807 08:34:57.280110 19399 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:57.280158 19399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:57.281734 19399 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 08:34:57.283066 19688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:57.283389 19399 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:34:57.283479 19399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/wal
uuid: "ea61347409eb4bfc818fa544931f8576"
format_stamp: "Formatted at 2026-08-07 08:34:57 on dist-test-slave-h3ft"
server_key: "c413e6972d943aac66e9e1c89201b78b"
server_key_iv: "121de507c3481553db52fa742523ca8b"
server_key_version: "encryptionkey@0"
I20260807 08:34:57.283581 19399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:57.303913 19399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:57.306519 19399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:57.312860 19399 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.60:45863
I20260807 08:34:57.313021 19768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.60:45863 every 8 connection(s)
I20260807 08:34:57.313608 19399 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/master-2/data/info.pb
I20260807 08:34:57.316103 19769 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:34:57.317636 19769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: Bootstrap starting.
I20260807 08:34:57.328764 19769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:34:57.329211 19769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: Bootstrap complete.
I20260807 08:34:57.330035 19769 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } }
I20260807 08:34:57.330197 19769 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ea61347409eb4bfc818fa544931f8576, State: Initialized, Role: LEARNER
I20260807 08:34:57.330317 19769 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } }
I20260807 08:34:57.330860 19769 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:57.330811 19773 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } } }
I20260807 08:34:57.331225 19773 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: This master's current role is: LEARNER
I20260807 08:34:57.336237 19789 catalog_manager.cc:1259] Loaded cluster ID: 52ca67c8727643308ba3e9a056ffe10c
I20260807 08:34:57.336323 19789 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: loading cluster ID for follower catalog manager: success
I20260807 08:34:57.339097 19789 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: acquiring CA information for follower catalog manager: success
I20260807 08:34:57.340910 19789 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 08:34:57.361908 19646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:57.362249 19646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:57.362349 19646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:57.367794 19646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:57.367889 19646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:57.367986 19646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:34:57.374527 19646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.1.keytab
--principal=oryx/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:41967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=45937
--tserver_master_addrs=127.9.174.62:42359,127.9.174.61:37673,127.9.174.60:45863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:57.376267 19646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:57.377758 19646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:34:57.385136 19230 heartbeater.cc:503] Master 127.9.174.62:42359 was elected leader, sending a full tablet report...
W20260807 08:34:57.392328 19800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:57.392468 19798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:57.392514 19794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:57.393589 19646 server_base.cc:1034] running on GCE node
I20260807 08:34:57.394261 19646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:57.395282 19646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:57.396502 19646 hybrid_clock.cc:648] HybridClock initialized: now 1786091697396479 us; error 37 us; skew 500 ppm
Aug 07 08:34:57 dist-test-slave-h3ft krb5kdc[18508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:57.400830 19646 init.cc:377] Logged in from keytab as oryx/127.9.174.1@KRBTEST.COM (short username oryx)
I20260807 08:34:57.402057 19646 webserver.cc:460] Webserver started at http://127.9.174.1:45937/ using document root <none> and password file <none>
I20260807 08:34:57.402812 19646 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:57.402957 19646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:57.408653 19646 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:34:57.411741 19808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:57.413535 19646 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:34:57.413725 19646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "f41ee8b27a214c348b0e0b86a2d74f54"
format_stamp: "Formatted at 2026-08-07 08:34:55 on dist-test-slave-h3ft"
server_key: "1e4539a6997bfdb0962e9c2fed69c300"
server_key_iv: "ae345df0c041121757da445da0828702"
server_key_version: "encryptionkey@0"
I20260807 08:34:57.414589 19646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:57.436266 19646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:57.440424 19646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:57.440696 19646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:57.441509 19646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:57.442593 19646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:57.442670 19646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:57.442750 19646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:57.442803 19646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:57.456178 19646 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:41967
I20260807 08:34:57.456198 19982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:41967 every 8 connection(s)
I20260807 08:34:57.457564 19646 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:34:57.460906 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 19646
I20260807 08:34:57.461190 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 18850
Aug 07 08:34:57 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM
Aug 07 08:34:57 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.61@KRBTEST.COM
Aug 07 08:34:57 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.60@KRBTEST.COM
I20260807 08:34:57.482064 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.460111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:59391 (local address 127.9.174.61:37673)
0807 08:34:57.460291 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.460296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.468468 (+ 8172us) server_negotiation.cc:407] Connection header received
0807 08:34:57.468665 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.468671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.468748 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.468804 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:57.469598 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.470634 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.471732 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.472025 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.472096 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.477307 (+ 5211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:57.477313 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:57.477319 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:57.477370 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:57.479261 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.480234 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.480238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.480239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.480337 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.480920 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.480923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.480925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.481171 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:57.481255 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.481707 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.481858 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 08:34:57.482064 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.460111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:48423 (local address 127.9.174.62:42359)
0807 08:34:57.460291 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.460296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.461781 (+ 1485us) server_negotiation.cc:407] Connection header received
0807 08:34:57.466110 (+ 4329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.466117 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.466199 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.466258 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:57.468517 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.469558 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.471493 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.471799 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.471891 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.476868 (+ 4977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:57.476873 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:57.476877 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:57.476916 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:57.479104 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.479920 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.479925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.479928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.480031 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.480513 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.480518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.480519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.480796 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:57.480899 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.481710 (+ 811us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.481858 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 08:34:57.483616 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.464773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:48037 (local address 127.9.174.60:45863)
0807 08:34:57.471696 (+ 6923us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.471702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.471717 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:34:57.471768 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.471771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.471833 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.471943 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:57.473659 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.474710 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.475770 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.476093 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.476207 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.479515 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:57.479521 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:57.479527 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:57.479568 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:57.481524 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.482078 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.482084 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.482089 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.482196 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.482549 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.482555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.482560 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.482824 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:57.482930 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.483254 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.483410 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6818,"thread_start_us":1701,"threads_started":1}
I20260807 08:34:57.483695 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:33389
--local_ip_for_outbound_sockets=127.9.174.2
--tserver_master_addrs=127.9.174.62:42359
--webserver_port=42405
--webserver_interface=127.9.174.2
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.2.keytab
--principal=oryx/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.174.62:42359,127.9.174.61:37673
--tserver_master_addrs=127.9.174.62:42359,127.9.174.61:37673,127.9.174.60:45863 with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:34:57.484406 19994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.460776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.1:48423)
0807 08:34:57.461551 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:57.461619 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:57.465809 (+ 4190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:57.466830 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:57.466844 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:57.467482 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:57.468282 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.468305 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.469763 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.469768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.471336 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.471365 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.472167 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.472171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.472475 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.473666 (+ 1191us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:57.473686 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:57.476541 (+ 2855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:57.479259 (+ 2718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.479266 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.479290 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.479789 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.480155 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.480159 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.480163 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.480382 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.480973 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:57.480983 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:57.481379 (+ 396us) client_negotiation.cc:769] Sending connection context
0807 08:34:57.482372 (+ 993us) client_negotiation.cc:240] Negotiation successful
0807 08:34:57.483389 (+ 1017us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":483,"spinlock_wait_cycles":8192,"thread_start_us":204,"threads_started":1}
I20260807 08:34:57.484946 19996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.462162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45863 (local address 127.9.174.1:48037)
0807 08:34:57.471318 (+ 9156us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:57.471335 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:57.471446 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:57.472094 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:57.472097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:57.472336 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:57.473481 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.473492 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.474837 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.474844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.475643 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.475653 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.476376 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.476381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.476452 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.477272 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:57.477292 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:57.479311 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:57.481657 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.481663 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.481668 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.481950 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.482310 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.482316 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.482321 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.482440 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.482917 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:57.482920 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:57.483033 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 08:34:57.483136 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 08:34:57.483267 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9078,"mutex_wait_us":967,"spinlock_wait_cycles":3328,"thread_start_us":2200,"threads_started":1}
I20260807 08:34:57.484406 19995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.460776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:37673 (local address 127.9.174.1:59391)
0807 08:34:57.468340 (+ 7564us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:57.468359 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:57.468555 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:57.468972 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:57.468977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:57.469246 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:57.469419 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.469430 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.470799 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.470802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.471586 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.471597 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.472269 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.472272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.472599 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.473428 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:57.473460 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:57.477118 (+ 3658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:57.479442 (+ 2324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.479445 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.479447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.479789 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.480458 (+ 669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.480463 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.480464 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.480805 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.481317 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:57.481320 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:57.481393 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 08:34:57.482667 (+ 1274us) client_negotiation.cc:240] Negotiation successful
0807 08:34:57.482908 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7424,"mutex_wait_us":65,"spinlock_wait_cycles":87552,"thread_start_us":189,"threads_started":1}
I20260807 08:34:57.486421 19990 heartbeater.cc:347] Connected to a master server at 127.9.174.61:37673
I20260807 08:34:57.486742 19990 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:57.486833 19983 heartbeater.cc:347] Connected to a master server at 127.9.174.60:45863
I20260807 08:34:57.486943 19983 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:57.487325 19984 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42359
I20260807 08:34:57.487393 19984 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:57.487537 19984 heartbeater.cc:511] Master 127.9.174.62:42359 requested a full tablet report, sending...
I20260807 08:34:57.487540 19983 heartbeater.cc:511] Master 127.9.174.60:45863 requested a full tablet report, sending...
I20260807 08:34:57.487723 19990 heartbeater.cc:511] Master 127.9.174.61:37673 requested a full tablet report, sending...
I20260807 08:34:57.489357 18567 ts_manager.cc:194] Re-registered known tserver with Master: f41ee8b27a214c348b0e0b86a2d74f54 (127.9.174.1:41967)
I20260807 08:34:57.489427 19439 ts_manager.cc:194] Registered new tserver with Master: f41ee8b27a214c348b0e0b86a2d74f54 (127.9.174.1:41967)
I20260807 08:34:57.490470 18567 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:48423
I20260807 08:34:57.492053 19712 ts_manager.cc:194] Registered new tserver with Master: f41ee8b27a214c348b0e0b86a2d74f54 (127.9.174.1:41967)
W20260807 08:34:57.636531 19998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:57.637048 19998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:57.637115 19998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:57.642437 19998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:57.642557 19998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:57.642642 19998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:34:57.649223 19998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.2.keytab
--principal=oryx/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:33389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=42405
--tserver_master_addrs=127.9.174.62:42359,127.9.174.61:37673,127.9.174.60:45863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:57.650719 19998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:57.652179 19998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:57.665138 20007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:57.665145 20008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:57.665275 19998 server_base.cc:1034] running on GCE node
W20260807 08:34:57.665560 20011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:57.666124 19998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:57.667055 19998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:57.668268 19998 hybrid_clock.cc:648] HybridClock initialized: now 1786091697668230 us; error 52 us; skew 500 ppm
Aug 07 08:34:57 dist-test-slave-h3ft krb5kdc[18508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:57.677927 19998 init.cc:377] Logged in from keytab as oryx/127.9.174.2@KRBTEST.COM (short username oryx)
I20260807 08:34:57.679358 19998 webserver.cc:460] Webserver started at http://127.9.174.2:42405/ using document root <none> and password file <none>
I20260807 08:34:57.680181 19998 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:57.680311 19998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:57.685546 19998 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:34:57.688614 20021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:57.690491 19998 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:34:57.690698 19998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "c02c956afe4c4c4f8ff1353b2e92beb4"
format_stamp: "Formatted at 2026-08-07 08:34:56 on dist-test-slave-h3ft"
server_key: "b64869bc7eac20f2880f8a3ab8586bdc"
server_key_iv: "bd020a305b713faa9009ff678feedc51"
server_key_version: "encryptionkey@0"
I20260807 08:34:57.691483 19998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:57.708076 19998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:57.710384 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.705543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45863 (local address 127.0.0.1:42226)
0807 08:34:57.705712 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:57.705730 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:57.705859 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:57.706224 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:57.706227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:57.706246 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:57.706371 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.706378 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.707502 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.707506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.708642 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.708655 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.709751 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.709758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.709917 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.709952 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 08:34:57.710056 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 08:34:57.710175 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 08:34:57.710368 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.705649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42226 (local address 127.9.174.60:45863)
0807 08:34:57.705790 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.705794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.705812 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:34:57.706018 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.706020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.706096 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.706186 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:57.706545 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.707367 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.708806 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.709598 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.709707 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.709866 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.710090 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.710186 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 08:34:57.711251 19730 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 08:34:57.711658 19773 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c7b45d28ba243aa8679d3071c58d4c9. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } } }
I20260807 08:34:57.711898 19773 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: This master's current role is: LEARNER
I20260807 08:34:57.713150 19998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:57.713399 19998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:57.714218 19998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:57.715452 19998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:57.715505 19998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:57.715553 19998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:57.715593 19998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:57.736294 19998 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:33389
I20260807 08:34:57.736327 20195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:33389 every 8 connection(s)
I20260807 08:34:57.738124 19998 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:34:57.747500 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 19998
I20260807 08:34:57.747773 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 19043
Aug 07 08:34:57 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.60@KRBTEST.COM
Aug 07 08:34:57 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM
Aug 07 08:34:57 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.61@KRBTEST.COM
I20260807 08:34:57.766777 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.743271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:50631 (local address 127.9.174.62:42359)
0807 08:34:57.743436 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.743441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.743461 (+ 20us) server_negotiation.cc:407] Connection header received
0807 08:34:57.744972 (+ 1511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.744978 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.745080 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.745156 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:57.747777 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.748661 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.754924 (+ 6263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.755311 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.755429 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.759881 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:57.759888 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:57.759895 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:57.759941 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:57.762925 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.764446 (+ 1521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.764452 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.764457 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.764566 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.765066 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.765072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.765075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.765391 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:57.765486 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.766404 (+ 918us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.766564 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:34:57.766777 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.744656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:38423 (local address 127.9.174.61:37673)
0807 08:34:57.744853 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.744862 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.744882 (+ 20us) server_negotiation.cc:407] Connection header received
0807 08:34:57.745073 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.745077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.745142 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.745208 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:57.747766 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.748876 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.754345 (+ 5469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.755241 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.755427 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.761760 (+ 6333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:57.761768 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:57.761774 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:57.761837 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:57.764795 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.765299 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.765303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.765304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.765394 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.765742 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.765745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.765746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.765984 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:57.766065 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.766404 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.766567 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 08:34:57.767078 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.744972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:39475 (local address 127.9.174.60:45863)
0807 08:34:57.745135 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:57.745140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:57.745776 (+ 636us) server_negotiation.cc:407] Connection header received
0807 08:34:57.745913 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:57.745918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:57.745998 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:57.746098 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:57.747746 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.748664 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.752832 (+ 4168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.753229 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.753330 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.760422 (+ 7092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:57.760428 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:57.760434 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:57.760475 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:57.762830 (+ 2355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.764404 (+ 1574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.764412 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.764416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.764574 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:57.765176 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:57.765184 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:57.765189 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:57.765574 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:57.765717 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:57.766505 (+ 788us) server_negotiation.cc:299] Negotiation successful
0807 08:34:57.766723 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 08:34:57.769116 20206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.741538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.2:50631)
0807 08:34:57.743048 (+ 1510us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:57.743127 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:57.744660 (+ 1533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:57.745606 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:57.745615 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:57.746468 (+ 853us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:57.747530 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.747555 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.752706 (+ 5151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.752712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.754760 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.754771 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.755612 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.755615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.755720 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.757289 (+ 1569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:57.757310 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:57.759658 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:57.763049 (+ 3391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.763063 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.763090 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.764313 (+ 1223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.764698 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.764702 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.764704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.764932 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.765590 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:57.765598 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:57.766066 (+ 468us) client_negotiation.cc:769] Sending connection context
0807 08:34:57.766994 (+ 928us) client_negotiation.cc:240] Negotiation successful
0807 08:34:57.767559 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1149,"mutex_wait_us":1484,"spinlock_wait_cycles":60416,"thread_start_us":890,"threads_started":1}
I20260807 08:34:57.769438 20209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.745237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45863 (local address 127.9.174.2:39475)
0807 08:34:57.745630 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:57.745646 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:57.745787 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:57.746270 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:57.746273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:57.746512 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:57.747535 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.747555 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.751102 (+ 3547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.751108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.752665 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.752678 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.753632 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.753638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.753926 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.756095 (+ 2169us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:57.756119 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:57.758959 (+ 2840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:57.763032 (+ 4073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.763058 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.763089 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.764219 (+ 1130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.764754 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.764761 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.764766 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.765009 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.765800 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:57.765805 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:57.766156 (+ 351us) client_negotiation.cc:769] Sending connection context
0807 08:34:57.767293 (+ 1137us) client_negotiation.cc:240] Negotiation successful
0807 08:34:57.767807 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":397,"spinlock_wait_cycles":14208,"thread_start_us":125,"threads_started":1}
I20260807 08:34:57.769819 20208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:57.743710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:37673 (local address 127.9.174.2:38423)
0807 08:34:57.744100 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:57.744116 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:57.744918 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:57.745607 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:57.745616 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:57.746526 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:57.747530 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.747556 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.752729 (+ 5173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.752736 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.754043 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:57.754061 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:57.755872 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:57.755882 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:57.756205 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:57.759301 (+ 3096us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:57.759327 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:57.761492 (+ 2165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:57.764920 (+ 3428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.764923 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.764925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.765175 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.765526 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:57.765530 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:57.765535 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:57.765628 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:57.766133 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:57.766136 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:57.766211 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 08:34:57.767365 (+ 1154us) client_negotiation.cc:240] Negotiation successful
0807 08:34:57.767661 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":224,"spinlock_wait_cycles":1408,"thread_start_us":210,"threads_started":1}
I20260807 08:34:57.773852 20201 heartbeater.cc:347] Connected to a master server at 127.9.174.61:37673
I20260807 08:34:57.773903 20200 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42359
I20260807 08:34:57.774169 20200 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:57.774227 20197 heartbeater.cc:347] Connected to a master server at 127.9.174.60:45863
I20260807 08:34:57.774168 20201 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:57.774317 20197 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:57.775081 20201 heartbeater.cc:511] Master 127.9.174.61:37673 requested a full tablet report, sending...
I20260807 08:34:57.776806 19439 ts_manager.cc:194] Registered new tserver with Master: c02c956afe4c4c4f8ff1353b2e92beb4 (127.9.174.2:33389)
I20260807 08:34:57.777426 20200 heartbeater.cc:511] Master 127.9.174.62:42359 requested a full tablet report, sending...
I20260807 08:34:57.777814 20197 heartbeater.cc:511] Master 127.9.174.60:45863 requested a full tablet report, sending...
I20260807 08:34:57.778290 18567 ts_manager.cc:194] Re-registered known tserver with Master: c02c956afe4c4c4f8ff1353b2e92beb4 (127.9.174.2:33389)
I20260807 08:34:57.778764 19712 ts_manager.cc:194] Registered new tserver with Master: c02c956afe4c4c4f8ff1353b2e92beb4 (127.9.174.2:33389)
I20260807 08:34:57.779249 18567 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:50631
I20260807 08:34:57.784296 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:36867
--local_ip_for_outbound_sockets=127.9.174.3
--tserver_master_addrs=127.9.174.62:42359
--webserver_port=38331
--webserver_interface=127.9.174.3
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.3.keytab
--principal=oryx/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.174.62:42359,127.9.174.61:37673
--tserver_master_addrs=127.9.174.62:42359,127.9.174.61:37673,127.9.174.60:45863 with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:57.938074 20212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:57.938498 20212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:57.938577 20212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:57.943192 20212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:57.943307 20212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:57.943413 20212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:34:57.949481 20212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.3.keytab
--principal=oryx/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:36867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=38331
--tserver_master_addrs=127.9.174.62:42359,127.9.174.61:37673,127.9.174.60:45863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:57.951061 20212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:57.952430 20212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:57.964457 20224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:57.964613 20223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:57.964613 20226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:57.964967 20212 server_base.cc:1034] running on GCE node
I20260807 08:34:57.965541 20212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:57.966374 20212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:57.967569 20212 hybrid_clock.cc:648] HybridClock initialized: now 1786091697967540 us; error 51 us; skew 500 ppm
Aug 07 08:34:57 dist-test-slave-h3ft krb5kdc[18508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:57.970921 20212 init.cc:377] Logged in from keytab as oryx/127.9.174.3@KRBTEST.COM (short username oryx)
I20260807 08:34:57.972129 20212 webserver.cc:460] Webserver started at http://127.9.174.3:38331/ using document root <none> and password file <none>
I20260807 08:34:57.972883 20212 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:57.973018 20212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:57.978001 20212 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:34:57.980798 20235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:57.982474 20212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 08:34:57.982641 20212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "f945358cf4aa423394dbbe1bbea16832"
format_stamp: "Formatted at 2026-08-07 08:34:56 on dist-test-slave-h3ft"
server_key: "4dbe4593e9716b090d5cd42daaccacb0"
server_key_iv: "3277cf45107969859b952027f3e05e81"
server_key_version: "encryptionkey@0"
I20260807 08:34:57.983518 20212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:58.018054 20212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:58.022709 20212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:58.022984 20212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:58.023882 20212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:34:58.024992 20212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:34:58.025049 20212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:58.025122 20212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:34:58.025156 20212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:58.036710 20212 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:36867
I20260807 08:34:58.036774 20397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:36867 every 8 connection(s)
I20260807 08:34:58.038074 20212 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:34:58.046275 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 20212
Aug 07 08:34:58 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM
Aug 07 08:34:58 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.61@KRBTEST.COM
Aug 07 08:34:58 dist-test-slave-h3ft krb5kdc[18508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091697, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.60@KRBTEST.COM
I20260807 08:34:58.059485 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.041247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:57101 (local address 127.9.174.62:42359)
0807 08:34:58.041446 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:58.041451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:58.041851 (+ 400us) server_negotiation.cc:407] Connection header received
0807 08:34:58.043203 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:58.043206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:58.043283 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:58.043366 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:58.045172 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.045765 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.048011 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.048383 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.048504 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.053106 (+ 4602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:58.053110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:58.053113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:58.053138 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:58.055963 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.056590 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.056594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.056596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.056694 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.057342 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.057349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.057353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.057646 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:58.057760 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:58.058683 (+ 923us) server_negotiation.cc:299] Negotiation successful
0807 08:34:58.058830 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 08:34:58.060650 20403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.040917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.3:57101)
0807 08:34:58.041639 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:58.041706 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:58.042917 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:58.043619 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:58.043627 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:58.044244 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:58.044995 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:58.045013 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.045957 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.045960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:58.047470 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:58.047482 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.048578 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.048583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:58.048811 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.049789 (+ 978us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:58.049807 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:58.052910 (+ 3103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:58.056089 (+ 3179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:58.056097 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:58.056124 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:58.056461 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:58.056783 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:58.056787 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:58.056789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:58.057005 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:58.057769 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:58.057780 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:58.058278 (+ 498us) client_negotiation.cc:769] Sending connection context
0807 08:34:58.058533 (+ 255us) client_negotiation.cc:240] Negotiation successful
0807 08:34:58.059225 (+ 692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":88,"spinlock_wait_cycles":11520,"thread_start_us":198,"threads_started":1}
I20260807 08:34:58.061659 20417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.049395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44054 (local address 127.9.174.61:37673)
0807 08:34:58.050038 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:58.050043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:58.050057 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:34:58.050104 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:58.050110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:58.050181 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:58.051481 (+ 1300us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:58.051618 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.052434 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.054305 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.054747 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.055290 (+ 543us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.056718 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:58.056722 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:58.056725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:58.056752 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:58.058636 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.059191 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.059195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.059197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.059300 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.059602 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.059605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.059606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.059859 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:58.060743 (+ 884us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:58.060836 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 08:34:58.061459 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":129,"threads_started":1}
I20260807 08:34:58.062430 20400 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42359
I20260807 08:34:58.062772 20400 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:58.063522 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.040368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:49171 (local address 127.9.174.61:37673)
0807 08:34:58.040605 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:58.040612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:58.041915 (+ 1303us) server_negotiation.cc:407] Connection header received
0807 08:34:58.043203 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:58.043207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:58.043287 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:58.043393 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:58.045968 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.046773 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.048101 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.048474 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.048579 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.053153 (+ 4574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:58.053157 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:58.053160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:58.053194 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:58.055595 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.061779 (+ 6184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.061785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.061788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.061879 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.062288 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.062294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.062295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.062576 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:58.062677 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:58.063176 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 08:34:58.063332 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 08:34:58.063664 20400 heartbeater.cc:511] Master 127.9.174.62:42359 requested a full tablet report, sending...
I20260807 08:34:58.064644 20406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.040920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:37673 (local address 127.9.174.3:49171)
0807 08:34:58.041639 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:58.041706 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:58.042995 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:58.043618 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:58.043626 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:58.044279 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:58.045006 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:58.045016 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.046915 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.046919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:58.047921 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:58.047934 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.048775 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.048777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:58.048853 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.049768 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:58.049803 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:58.052910 (+ 3107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:58.057489 (+ 4579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:58.057492 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:58.057495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:58.061653 (+ 4158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:58.062011 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:58.062032 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:58.062037 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:58.062152 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:58.062754 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:58.062756 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:58.062896 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 08:34:58.063683 (+ 787us) client_negotiation.cc:240] Negotiation successful
0807 08:34:58.063905 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"mutex_wait_us":189,"spinlock_wait_cycles":96768,"thread_start_us":198,"threads_started":1}
I20260807 08:34:58.065295 18567 ts_manager.cc:194] Re-registered known tserver with Master: f945358cf4aa423394dbbe1bbea16832 (127.9.174.3:36867)
I20260807 08:34:58.065371 20401 heartbeater.cc:347] Connected to a master server at 127.9.174.61:37673
I20260807 08:34:58.065469 20401 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:58.065711 20401 heartbeater.cc:511] Master 127.9.174.61:37673 requested a full tablet report, sending...
I20260807 08:34:58.066226 18567 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:57101
I20260807 08:34:58.066583 19439 ts_manager.cc:194] Registered new tserver with Master: f945358cf4aa423394dbbe1bbea16832 (127.9.174.3:36867)
I20260807 08:34:58.066955 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.042536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:40739 (local address 127.9.174.60:45863)
0807 08:34:58.042687 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:58.042695 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:58.042761 (+ 66us) server_negotiation.cc:407] Connection header received
0807 08:34:58.043575 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:58.043579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:58.043653 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:58.043755 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:58.046963 (+ 3208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.047779 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.051507 (+ 3728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.051926 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.052072 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.061653 (+ 9581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:58.061658 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:58.061661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:58.061701 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:58.064014 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.065214 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.065218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.065220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.065331 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.065802 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.065805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.065808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.066049 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:58.066132 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:58.066658 (+ 526us) server_negotiation.cc:299] Negotiation successful
0807 08:34:58.066822 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 08:34:58.067606 20409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.042352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45863 (local address 127.9.174.3:40739)
0807 08:34:58.042665 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:58.042680 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:58.042925 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:58.043796 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:58.043800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:58.044270 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:58.045004 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:58.045016 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.047952 (+ 2936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.047958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:58.051347 (+ 3389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:58.051361 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.053144 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.053147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:58.053284 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.055798 (+ 2514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:58.055818 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:58.061410 (+ 5592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:58.064381 (+ 2971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:58.064384 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:58.064386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:58.065090 (+ 704us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:58.065560 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:58.065566 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:58.065567 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:58.065667 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:58.066274 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:58.066277 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:58.066434 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 08:34:58.067270 (+ 836us) client_negotiation.cc:240] Negotiation successful
0807 08:34:58.067452 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":56,"spinlock_wait_cycles":12672,"thread_start_us":113,"threads_started":1}
I20260807 08:34:58.069622 20399 heartbeater.cc:347] Connected to a master server at 127.9.174.60:45863
I20260807 08:34:58.069737 20399 heartbeater.cc:464] Registering TS with master...
I20260807 08:34:58.070011 20399 heartbeater.cc:511] Master 127.9.174.60:45863 requested a full tablet report, sending...
I20260807 08:34:58.070977 19712 ts_manager.cc:194] Registered new tserver with Master: f945358cf4aa423394dbbe1bbea16832 (127.9.174.3:36867)
I20260807 08:34:58.071560 20419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.050116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.9.174.62:42359)
0807 08:34:58.052841 (+ 2725us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:58.052847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:58.052861 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:34:58.052906 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:58.052909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:58.052962 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:58.053065 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:58.054950 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.055832 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.059279 (+ 3447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.059685 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.059784 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.062638 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:58.062641 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:58.062644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:58.062672 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:58.068948 (+ 6276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.069806 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.069812 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.069816 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.069917 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.070429 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.070433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.070435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.070673 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:58.070845 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:58.071226 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 08:34:58.071378 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2621,"thread_start_us":107,"threads_started":1}
I20260807 08:34:58.072364 20420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.050370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42240 (local address 127.9.174.60:45863)
0807 08:34:58.051665 (+ 1295us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:58.051670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:58.052344 (+ 674us) server_negotiation.cc:407] Connection header received
0807 08:34:58.052394 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:58.052397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:58.052452 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:58.052559 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:58.056016 (+ 3457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.056890 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.060367 (+ 3477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.060767 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.060857 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.068207 (+ 7350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:58.068211 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:58.068213 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:58.068236 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:58.069895 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.070424 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.070427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.070429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.070527 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.070861 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.070865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.070867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.071156 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:58.071288 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:58.072110 (+ 822us) server_negotiation.cc:299] Negotiation successful
0807 08:34:58.072234 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1194,"thread_start_us":144,"threads_started":1}
I20260807 08:34:58.074684 18567 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:34:58.078416 18567 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:34:58.082093 19662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: NON_VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: true } } }
I20260807 08:34:58.082326 19662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:58.084821 19662 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9: attempting to promote NON_VOTER ea61347409eb4bfc818fa544931f8576 to VOTER
I20260807 08:34:58.085295 19662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } }
I20260807 08:34:58.086745 19730 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 LEARNER]: Refusing update from remote peer 5c7b45d28ba243aa8679d3071c58d4c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 08:34:58.086755 19456 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 FOLLOWER]: Refusing update from remote peer 5c7b45d28ba243aa8679d3071c58d4c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 08:34:58.087189 19662 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 08:34:58.087350 20423 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 08:34:58.089843 20425 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ea61347409eb4bfc818fa544931f8576 (127.9.174.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } } }
I20260807 08:34:58.090236 19730 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ea61347409eb4bfc818fa544931f8576 (127.9.174.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } } }
I20260807 08:34:58.090324 19456 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ea61347409eb4bfc818fa544931f8576 (127.9.174.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } } }
I20260807 08:34:58.092002 20425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } } }
I20260807 08:34:58.092142 20425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:58.092154 19773 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } } }
I20260807 08:34:58.092280 19773 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:58.092305 20425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } } }
I20260807 08:34:58.092379 19662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } } }
I20260807 08:34:58.092507 19662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:58.092393 20425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:58.093024 20439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } } }
I20260807 08:34:58.093178 20439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:58.109573 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.102535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:33389 (local address 127.0.0.1:43632)
0807 08:34:58.102748 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:58.102766 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:58.102944 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:58.104820 (+ 1876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:58.104823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:58.104844 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:58.105022 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:58.105032 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.106715 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.106719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:58.107980 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:58.107989 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.108993 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.108997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:58.109119 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.109181 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 08:34:58.109280 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 08:34:58.109371 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":75}
I20260807 08:34:58.109649 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.102769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43632 (local address 127.9.174.2:33389)
0807 08:34:58.104074 (+ 1305us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:58.104085 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:58.104137 (+ 52us) server_negotiation.cc:407] Connection header received
0807 08:34:58.104260 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:58.104268 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:58.104607 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:58.105207 (+ 600us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:58.105418 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.106570 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.108145 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.108841 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.108948 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.109161 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:58.109333 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 08:34:58.109465 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1163,"thread_start_us":161,"threads_started":1}
I20260807 08:34:58.110774 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.102699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:36867 (local address 127.0.0.1:56336)
0807 08:34:58.104002 (+ 1303us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:58.104018 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:58.104155 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:58.104832 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:58.104838 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:58.104863 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:58.105025 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:58.105032 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.106663 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.106667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:58.107999 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:58.108012 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.110339 (+ 2327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.110344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:58.110438 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.110465 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 08:34:58.110550 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 08:34:58.110628 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1236,"mutex_wait_us":93,"thread_start_us":179,"threads_started":1}
I20260807 08:34:58.110898 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.103372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56336 (local address 127.9.174.3:36867)
0807 08:34:58.103806 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:58.103855 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:58.104157 (+ 302us) server_negotiation.cc:407] Connection header received
0807 08:34:58.104432 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:58.104437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:58.104604 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:58.104829 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:58.105163 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.106515 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.109237 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.110183 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.110321 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.110491 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:58.110596 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 08:34:58.110710 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":178,"threads_started":1}
I20260807 08:34:58.112243 20440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.103440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:41967 (local address 127.0.0.1:35494)
0807 08:34:58.106890 (+ 3450us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:58.106906 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:58.107043 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:58.107927 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:58.107931 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:58.107950 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:58.108091 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:58.108098 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.109114 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.109118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:58.110406 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:58.110417 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.111795 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.111798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:58.111902 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.111929 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 08:34:58.112004 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 08:34:58.112084 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3369,"mutex_wait_us":1,"thread_start_us":146,"threads_started":1}
I20260807 08:34:58.112773 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.103661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35494 (local address 127.9.174.1:41967)
0807 08:34:58.104463 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:58.104470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:58.107168 (+ 2698us) server_negotiation.cc:407] Connection header received
0807 08:34:58.107286 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:58.107293 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:58.107587 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:58.107742 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:58.108193 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.108976 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.110561 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.111452 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.111561 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.111719 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:58.112461 (+ 742us) server_negotiation.cc:299] Negotiation successful
0807 08:34:58.112610 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":124,"threads_started":1}
I20260807 08:34:58.114429 20092 tablet_service.cc:1467] Processing CreateTablet for tablet fba0ecaaaceb447db364c29381f76f88 (DEFAULT_TABLE table=test-table [id=904e9bf7324d4653b10bcbaaf264ff5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:58.114426 20299 tablet_service.cc:1467] Processing CreateTablet for tablet fba0ecaaaceb447db364c29381f76f88 (DEFAULT_TABLE table=test-table [id=904e9bf7324d4653b10bcbaaf264ff5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:58.115906 20092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fba0ecaaaceb447db364c29381f76f88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:58.115945 20299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fba0ecaaaceb447db364c29381f76f88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:58.116107 19884 tablet_service.cc:1467] Processing CreateTablet for tablet fba0ecaaaceb447db364c29381f76f88 (DEFAULT_TABLE table=test-table [id=904e9bf7324d4653b10bcbaaf264ff5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:34:58.117292 19884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fba0ecaaaceb447db364c29381f76f88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:58.125226 20450 tablet_bootstrap.cc:492] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832: Bootstrap starting.
I20260807 08:34:58.126732 20451 tablet_bootstrap.cc:492] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4: Bootstrap starting.
I20260807 08:34:58.127398 20452 tablet_bootstrap.cc:492] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54: Bootstrap starting.
I20260807 08:34:58.128562 20450 tablet_bootstrap.cc:654] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:58.129112 20451 tablet_bootstrap.cc:654] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:58.129499 20452 tablet_bootstrap.cc:654] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:58.129539 20450 log.cc:824] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:58.130044 20451 log.cc:824] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:58.130393 20452 log.cc:824] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:58.132562 20451 tablet_bootstrap.cc:492] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4: No bootstrap required, opened a new log
I20260807 08:34:58.132562 20450 tablet_bootstrap.cc:492] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832: No bootstrap required, opened a new log
I20260807 08:34:58.132658 20452 tablet_bootstrap.cc:492] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54: No bootstrap required, opened a new log
I20260807 08:34:58.132861 20451 ts_tablet_manager.cc:1397] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.002s
I20260807 08:34:58.132881 20452 ts_tablet_manager.cc:1397] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 08:34:58.132959 20450 ts_tablet_manager.cc:1397] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 08:34:58.137163 20452 raft_consensus.cc:348] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } } peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } }
I20260807 08:34:58.137516 20452 raft_consensus.cc:374] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:58.137583 20452 raft_consensus.cc:729] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41ee8b27a214c348b0e0b86a2d74f54, State: Initialized, Role: FOLLOWER
I20260807 08:34:58.137763 20451 raft_consensus.cc:348] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } } peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } }
I20260807 08:34:58.138053 20451 raft_consensus.cc:374] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:58.138114 20451 raft_consensus.cc:729] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c02c956afe4c4c4f8ff1353b2e92beb4, State: Initialized, Role: FOLLOWER
I20260807 08:34:58.138232 20452 consensus_queue.cc:260] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } } peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } }
I20260807 08:34:58.138258 20450 raft_consensus.cc:348] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } } peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } }
I20260807 08:34:58.138533 20450 raft_consensus.cc:374] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:34:58.138602 20450 raft_consensus.cc:729] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f945358cf4aa423394dbbe1bbea16832, State: Initialized, Role: FOLLOWER
I20260807 08:34:58.138545 20451 consensus_queue.cc:260] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } } peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } }
I20260807 08:34:58.139127 20450 consensus_queue.cc:260] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } } peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } }
I20260807 08:34:58.139623 20452 ts_tablet_manager.cc:1428] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 08:34:58.139654 20451 ts_tablet_manager.cc:1428] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20260807 08:34:58.139863 20200 heartbeater.cc:503] Master 127.9.174.62:42359 was elected leader, sending a full tablet report...
I20260807 08:34:58.140101 20450 ts_tablet_manager.cc:1428] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 08:34:58.141080 19984 heartbeater.cc:503] Master 127.9.174.62:42359 was elected leader, sending a full tablet report...
I20260807 08:34:58.143205 20400 heartbeater.cc:503] Master 127.9.174.62:42359 was elected leader, sending a full tablet report...
I20260807 08:34:58.194854 20460 raft_consensus.cc:482] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:34:58.195276 20460 raft_consensus.cc:504] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } } peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } }
I20260807 08:34:58.196933 20460 leader_election.cc:288] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c02c956afe4c4c4f8ff1353b2e92beb4 (127.9.174.2:33389), f41ee8b27a214c348b0e0b86a2d74f54 (127.9.174.1:41967)
I20260807 08:34:58.202908 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.197497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:39937 (local address 127.9.174.2:33389)
0807 08:34:58.197799 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:58.197805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:58.197822 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:34:58.197897 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:58.197900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:58.197962 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:58.198055 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:58.198531 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.199426 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.201531 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.202192 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.202302 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.202389 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:58.202643 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 08:34:58.202735 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187}
I20260807 08:34:58.202908 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.197440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:47413 (local address 127.9.174.1:41967)
0807 08:34:58.197613 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:58.197619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:58.197798 (+ 179us) server_negotiation.cc:407] Connection header received
0807 08:34:58.197897 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:58.197900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:58.197962 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:58.198048 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:58.198544 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.199416 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.201531 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.202177 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.202302 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.202389 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:58.202643 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 08:34:58.202735 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 08:34:58.202968 20409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.197267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:33389 (local address 127.9.174.3:39937)
0807 08:34:58.197435 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:58.197457 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:58.197640 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:58.198090 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:58.198094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:58.198127 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:58.198395 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:58.198408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.199583 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.199586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:58.201362 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:58.201375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.202324 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.202328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:58.202489 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.202521 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 08:34:58.202606 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 08:34:58.202731 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":269,"spinlock_wait_cycles":6272}
I20260807 08:34:58.202968 20406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.197270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:41967 (local address 127.9.174.3:47413)
0807 08:34:58.197666 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:58.197704 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:58.197806 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:58.198090 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:58.198093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:58.198129 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:58.198393 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:58.198403 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.199581 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.199585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:58.201361 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:58.201375 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.202324 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.202328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:58.202473 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.202526 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 08:34:58.202606 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 08:34:58.202731 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":637,"spinlock_wait_cycles":4992}
I20260807 08:34:58.204109 20098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba0ecaaaceb447db364c29381f76f88" candidate_uuid: "f945358cf4aa423394dbbe1bbea16832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" is_pre_election: true
I20260807 08:34:58.204264 19917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba0ecaaaceb447db364c29381f76f88" candidate_uuid: "f945358cf4aa423394dbbe1bbea16832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" is_pre_election: true
I20260807 08:34:58.204609 20098 raft_consensus.cc:2454] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f945358cf4aa423394dbbe1bbea16832 in term 0.
I20260807 08:34:58.204640 19917 raft_consensus.cc:2454] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f945358cf4aa423394dbbe1bbea16832 in term 0.
I20260807 08:34:58.205260 20239 leader_election.cc:302] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: c02c956afe4c4c4f8ff1353b2e92beb4, f41ee8b27a214c348b0e0b86a2d74f54, f945358cf4aa423394dbbe1bbea16832; no voters:
I20260807 08:34:58.205659 20460 raft_consensus.cc:2781] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:34:58.205755 20460 raft_consensus.cc:482] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:34:58.205811 20460 raft_consensus.cc:3037] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:58.207324 20460 raft_consensus.cc:504] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } } peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } }
I20260807 08:34:58.207744 20460 leader_election.cc:288] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [CANDIDATE]: Term 1 election: Requested vote from peers c02c956afe4c4c4f8ff1353b2e92beb4 (127.9.174.2:33389), f41ee8b27a214c348b0e0b86a2d74f54 (127.9.174.1:41967)
I20260807 08:34:58.208199 20098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba0ecaaaceb447db364c29381f76f88" candidate_uuid: "f945358cf4aa423394dbbe1bbea16832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4"
I20260807 08:34:58.208267 19917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba0ecaaaceb447db364c29381f76f88" candidate_uuid: "f945358cf4aa423394dbbe1bbea16832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41ee8b27a214c348b0e0b86a2d74f54"
I20260807 08:34:58.208364 20098 raft_consensus.cc:3037] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:58.208439 19917 raft_consensus.cc:3037] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:58.209733 19917 raft_consensus.cc:2454] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f945358cf4aa423394dbbe1bbea16832 in term 1.
W20260807 08:34:58.209822 19991 tablet.cc:2369] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:34:58.210098 20236 leader_election.cc:302] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f41ee8b27a214c348b0e0b86a2d74f54, f945358cf4aa423394dbbe1bbea16832; no voters:
I20260807 08:34:58.210314 20460 raft_consensus.cc:2781] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:58.210402 20098 raft_consensus.cc:2454] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f945358cf4aa423394dbbe1bbea16832 in term 1.
I20260807 08:34:58.210619 20460 raft_consensus.cc:686] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [term 1 LEADER]: Becoming Leader. State: Replica: f945358cf4aa423394dbbe1bbea16832, State: Running, Role: LEADER
I20260807 08:34:58.211099 20460 consensus_queue.cc:237] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } } peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } }
I20260807 08:34:58.215441 18572 catalog_manager.cc:5495] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 reported cstate change: term changed from 0 to 1, leader changed from <none> to f945358cf4aa423394dbbe1bbea16832 (127.9.174.3). New cstate: current_term: 1 leader_uuid: "f945358cf4aa423394dbbe1bbea16832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } health_report { overall_health: UNKNOWN } } }
I20260807 08:34:58.239246 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.234757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56340 (local address 127.9.174.3:36867)
0807 08:34:58.235007 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:58.235047 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:58.235072 (+ 25us) server_negotiation.cc:407] Connection header received
0807 08:34:58.235248 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:58.235251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:58.235340 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:58.235464 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:58.235783 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.236359 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.237223 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.237457 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.237591 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.237959 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:58.238532 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:58.238635 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:58.238851 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 08:34:58.238937 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
W20260807 08:34:58.244453 20204 tablet.cc:2369] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:34:58.247025 20098 raft_consensus.cc:1261] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 1 FOLLOWER]: Refusing update from remote peer f945358cf4aa423394dbbe1bbea16832: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:58.247062 19917 raft_consensus.cc:1261] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54 [term 1 FOLLOWER]: Refusing update from remote peer f945358cf4aa423394dbbe1bbea16832: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:58.248059 20462 consensus_queue.cc:1035] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:58.248282 20460 consensus_queue.cc:1035] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:58.264248 20467 mvcc.cc:205] Tried to move back new op lower bound from 7315831596009889792 to 7315831595875794944. Current Snapshot: MvccSnapshot[applied={T|T < 7315831596009889792}]
I20260807 08:34:58.276299 18572 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:58.276577 18572 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:34:58.281948 18572 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c7b45d28ba243aa8679d3071c58d4c9: Sending DeleteTablet for 3 replicas of tablet fba0ecaaaceb447db364c29381f76f88
I20260807 08:34:58.282905 20092 tablet_service.cc:1514] Processing DeleteTablet for tablet fba0ecaaaceb447db364c29381f76f88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:58 UTC) from {username='oryx', principal='oryx/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:43632
I20260807 08:34:58.283370 20299 tablet_service.cc:1514] Processing DeleteTablet for tablet fba0ecaaaceb447db364c29381f76f88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:58 UTC) from {username='oryx', principal='oryx/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:56336
I20260807 08:34:58.283411 19884 tablet_service.cc:1514] Processing DeleteTablet for tablet fba0ecaaaceb447db364c29381f76f88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:34:58 UTC) from {username='oryx', principal='oryx/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:35494
I20260807 08:34:58.283779 20481 tablet_replica.cc:330] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4: stopping tablet replica
I20260807 08:34:58.284088 20482 tablet_replica.cc:330] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54: stopping tablet replica
I20260807 08:34:58.284454 20482 raft_consensus.cc:2229] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:34:58.284454 20481 raft_consensus.cc:2229] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:34:58.284916 20481 raft_consensus.cc:2258] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:58.284914 20482 raft_consensus.cc:2258] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:58.287135 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 18528
I20260807 08:34:58.287469 20483 tablet_replica.cc:330] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832: stopping tablet replica
I20260807 08:34:58.293455 20483 raft_consensus.cc:2229] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:34:58.294098 20483 raft_consensus.cc:2258] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:34:58.296793 20483 ts_tablet_manager.cc:1905] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:58.297276 20482 ts_tablet_manager.cc:1905] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:58.300627 20483 ts_tablet_manager.cc:1918] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:34:58.300765 20483 log.cc:1196] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/fba0ecaaaceb447db364c29381f76f88
I20260807 08:34:58.301084 20481 ts_tablet_manager.cc:1905] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:34:58.301170 20483 ts_tablet_manager.cc:1939] T fba0ecaaaceb447db364c29381f76f88 P f945358cf4aa423394dbbe1bbea16832: Deleting consensus metadata
I20260807 08:34:58.303808 20482 ts_tablet_manager.cc:1918] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:34:58.303960 20482 log.cc:1196] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-0/wal/wals/fba0ecaaaceb447db364c29381f76f88
I20260807 08:34:58.304342 20482 ts_tablet_manager.cc:1939] T fba0ecaaaceb447db364c29381f76f88 P f41ee8b27a214c348b0e0b86a2d74f54: Deleting consensus metadata
I20260807 08:34:58.306368 20481 ts_tablet_manager.cc:1918] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:34:58.306581 20481 log.cc:1196] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/fba0ecaaaceb447db364c29381f76f88
I20260807 08:34:58.307044 20481 ts_tablet_manager.cc:1939] T fba0ecaaaceb447db364c29381f76f88 P c02c956afe4c4c4f8ff1353b2e92beb4: Deleting consensus metadata
W20260807 08:34:58.312641 19815 connection.cc:535] server connection from 127.0.0.1:35494 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 08:34:58.312662 20239 connection.cc:535] server connection from 127.0.0.1:56336 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 08:34:58.312714 20023 connection.cc:535] server connection from 127.0.0.1:43632 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:34:58.322222 20417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.285920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44060 (local address 127.9.174.61:37673)
0807 08:34:58.286657 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:58.286663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:58.286679 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:34:58.286726 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:58.286729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:58.286783 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:58.286885 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:58.307621 (+ 20736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.308579 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.309912 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.310398 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.310555 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.317211 (+ 6656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:58.317216 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:58.317219 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:58.317265 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:58.319325 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.319949 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.319953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.319955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.320071 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.320448 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.320452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.320453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.320703 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:58.320836 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:58.321883 (+ 1047us) server_negotiation.cc:299] Negotiation successful
0807 08:34:58.322034 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631}
I20260807 08:34:58.329452 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.317892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44064 (local address 127.9.174.61:37673)
0807 08:34:58.318030 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:58.318034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:58.319124 (+ 1090us) server_negotiation.cc:407] Connection header received
0807 08:34:58.319290 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:58.319293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:58.319343 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:58.319398 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:58.320748 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.321604 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.322930 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.323325 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.323402 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.325293 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:58.325298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:58.325300 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:58.325327 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:58.327364 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.327899 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.327903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.327905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.327997 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.328300 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.328303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.328305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.328541 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:58.328659 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:58.329094 (+ 435us) server_negotiation.cc:299] Negotiation successful
0807 08:34:58.329242 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:34:58.331212 20420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.287122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42250 (local address 127.9.174.60:45863)
0807 08:34:58.287245 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:58.287251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:58.321539 (+ 34288us) server_negotiation.cc:407] Connection header received
0807 08:34:58.321637 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:58.321643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:58.321705 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:58.321794 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:58.322685 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.323510 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.324939 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.325308 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.325415 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.327032 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:58.327036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:58.327039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:58.327071 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:58.328937 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.329447 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.329450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.329453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.329912 (+ 459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.330227 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.330231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.330232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.330472 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:58.330619 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:58.330868 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 08:34:58.331042 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 08:34:58.337646 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:58.317299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42254 (local address 127.9.174.60:45863)
0807 08:34:58.317482 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:58.317488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:58.318736 (+ 1248us) server_negotiation.cc:407] Connection header received
0807 08:34:58.318881 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:58.318897 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:58.318967 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:58.319095 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:58.327942 (+ 8847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.328822 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.330125 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:58.330459 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:58.330550 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:58.333528 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:58.333532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:58.333535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:58.333562 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:58.335429 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.335963 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.335967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.335969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.336065 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:58.336363 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:58.336366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:58.336368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:58.336623 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:58.336741 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:58.337205 (+ 464us) server_negotiation.cc:299] Negotiation successful
0807 08:34:58.337390 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 08:34:59.265172 20496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:59.264512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.3:47627)
0807 08:34:59.264901 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:59.265018 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":97,"threads_started":1}
I20260807 08:34:59.265213 20497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:59.264494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.2:43943)
0807 08:34:59.264884 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:59.265025 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":107,"threads_started":1}
I20260807 08:34:59.265625 20499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:59.265015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.1:37629)
0807 08:34:59.265381 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:59.265501 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":103,"threads_started":1}
W20260807 08:34:59.266072 19984 heartbeater.cc:644] Failed to heartbeat to 127.9.174.62:42359 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
W20260807 08:34:59.266072 20200 heartbeater.cc:644] Failed to heartbeat to 127.9.174.62:42359 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
W20260807 08:34:59.266250 20400 heartbeater.cc:644] Failed to heartbeat to 127.9.174.62:42359 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
I20260807 08:34:59.782653 20500 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c7b45d28ba243aa8679d3071c58d4c9)
I20260807 08:34:59.782929 20502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c7b45d28ba243aa8679d3071c58d4c9)
I20260807 08:34:59.783097 20500 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } }
I20260807 08:34:59.783274 20502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } }
I20260807 08:34:59.784557 20502 leader_election.cc:288] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c7b45d28ba243aa8679d3071c58d4c9 (127.9.174.62:42359), ea61347409eb4bfc818fa544931f8576 (127.9.174.60:45863)
I20260807 08:34:59.784672 20500 leader_election.cc:288] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c7b45d28ba243aa8679d3071c58d4c9 (127.9.174.62:42359), 7438c78b2f514c20babb5aae6e8bea9e (127.9.174.61:37673)
I20260807 08:34:59.785761 20504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:59.785054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:38664)
0807 08:34:59.785477 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:59.785597 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":87,"threads_started":1}
I20260807 08:34:59.785935 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:59.784872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:38656)
0807 08:34:59.785684 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:59.785793 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":120,"threads_started":1}
W20260807 08:34:59.786013 19691 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
W20260807 08:34:59.786317 19417 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
I20260807 08:34:59.787447 20504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:59.787133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:38676)
0807 08:34:59.787295 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:59.787351 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 08:34:59.787576 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:59.787293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:38692)
0807 08:34:59.787446 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:59.787499 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260807 08:34:59.787640 19691 leader_election.cc:334] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5c7b45d28ba243aa8679d3071c58d4c9 (127.9.174.62:42359): Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
W20260807 08:34:59.787895 19417 leader_election.cc:334] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5c7b45d28ba243aa8679d3071c58d4c9 (127.9.174.62:42359): Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
I20260807 08:34:59.790726 20506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:59.785206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49052 (local address 127.9.174.61:37673)
0807 08:34:59.785526 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:59.785532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:59.785754 (+ 222us) server_negotiation.cc:407] Connection header received
0807 08:34:59.786190 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:59.786194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:59.786254 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:59.786329 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:59.786839 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:59.787730 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:59.789347 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:59.790057 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:59.790151 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:59.790205 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:59.790488 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 08:34:59.790592 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":102,"threads_started":1}
I20260807 08:34:59.790751 20505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:59.785109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:37673 (local address 127.0.0.1:49052)
0807 08:34:59.785645 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:59.785661 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:59.786004 (+ 343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:59.786542 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:59.786545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:59.786564 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:59.786695 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:59.786702 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:59.787892 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:59.787899 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:59.789172 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:59.789183 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:59.790188 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:59.790192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:59.790288 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:59.790357 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 08:34:59.790444 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 08:34:59.790552 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":26,"thread_start_us":90,"threads_started":1}
I20260807 08:34:59.790987 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:59.785566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47594 (local address 127.9.174.60:45863)
0807 08:34:59.785795 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:59.785800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:59.786033 (+ 233us) server_negotiation.cc:407] Connection header received
0807 08:34:59.786298 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:59.786301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:59.786367 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:59.786434 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:34:59.786969 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:59.788024 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:59.789801 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:59.790352 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:59.790579 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:59.790644 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:59.790783 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 08:34:59.790878 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":147,"thread_start_us":57,"threads_started":1}
I20260807 08:34:59.791147 20507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:59.785305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45863 (local address 127.0.0.1:47594)
0807 08:34:59.785938 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:59.785955 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:59.786133 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:59.786634 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:59.786639 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:59.786666 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:34:59.786807 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:59.786816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:59.788300 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:59.788306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:59.789566 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:59.789577 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:59.790562 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:59.790566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:59.790650 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:59.790673 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:34:59.790771 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 08:34:59.791030 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":80,"threads_started":1}
I20260807 08:34:59.791890 19730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7438c78b2f514c20babb5aae6e8bea9e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ea61347409eb4bfc818fa544931f8576" is_pre_election: true
I20260807 08:34:59.791879 19456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea61347409eb4bfc818fa544931f8576" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7438c78b2f514c20babb5aae6e8bea9e" is_pre_election: true
I20260807 08:34:59.792166 19730 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7438c78b2f514c20babb5aae6e8bea9e in term 1.
I20260807 08:34:59.792200 19456 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea61347409eb4bfc818fa544931f8576 in term 1.
I20260807 08:34:59.792608 19417 leader_election.cc:302] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7438c78b2f514c20babb5aae6e8bea9e, ea61347409eb4bfc818fa544931f8576; no voters: 5c7b45d28ba243aa8679d3071c58d4c9
I20260807 08:34:59.792639 19692 leader_election.cc:302] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7438c78b2f514c20babb5aae6e8bea9e, ea61347409eb4bfc818fa544931f8576; no voters: 5c7b45d28ba243aa8679d3071c58d4c9
I20260807 08:34:59.792903 20502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 08:34:59.792977 20502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c7b45d28ba243aa8679d3071c58d4c9)
I20260807 08:34:59.793020 20502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:34:59.794234 20502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } }
I20260807 08:34:59.794710 20502 leader_election.cc:288] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [CANDIDATE]: Term 2 election: Requested vote from peers 5c7b45d28ba243aa8679d3071c58d4c9 (127.9.174.62:42359), ea61347409eb4bfc818fa544931f8576 (127.9.174.60:45863)
I20260807 08:34:59.794888 20500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 08:34:59.794965 20500 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c7b45d28ba243aa8679d3071c58d4c9)
I20260807 08:34:59.795074 20500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:34:59.795270 20507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:59.794848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:38700)
0807 08:34:59.795081 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:59.795163 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 08:34:59.795329 19730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7438c78b2f514c20babb5aae6e8bea9e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ea61347409eb4bfc818fa544931f8576"
I20260807 08:34:59.796043 20507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:59.795802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:38712)
0807 08:34:59.795921 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:59.795979 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 08:34:59.796164 19417 leader_election.cc:334] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5c7b45d28ba243aa8679d3071c58d4c9 (127.9.174.62:42359): Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
I20260807 08:34:59.796512 20500 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } }
I20260807 08:34:59.796774 19730 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7438c78b2f514c20babb5aae6e8bea9e in current term 2: Already voted for candidate ea61347409eb4bfc818fa544931f8576 in this term.
I20260807 08:34:59.796926 20500 leader_election.cc:288] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [CANDIDATE]: Term 2 election: Requested vote from peers 5c7b45d28ba243aa8679d3071c58d4c9 (127.9.174.62:42359), 7438c78b2f514c20babb5aae6e8bea9e (127.9.174.61:37673)
I20260807 08:34:59.797317 20505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:59.797006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:38714)
0807 08:34:59.797183 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:59.797235 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260807 08:34:59.797407 19456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea61347409eb4bfc818fa544931f8576" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7438c78b2f514c20babb5aae6e8bea9e"
I20260807 08:34:59.797494 19417 leader_election.cc:302] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7438c78b2f514c20babb5aae6e8bea9e; no voters: 5c7b45d28ba243aa8679d3071c58d4c9, ea61347409eb4bfc818fa544931f8576
I20260807 08:34:59.797652 19456 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ea61347409eb4bfc818fa544931f8576 in current term 2: Already voted for candidate 7438c78b2f514c20babb5aae6e8bea9e in this term.
I20260807 08:34:59.798009 20502 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 08:34:59.798671 20505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:59.798444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:38720)
0807 08:34:59.798560 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:59.798611 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 08:34:59.798784 19691 leader_election.cc:334] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5c7b45d28ba243aa8679d3071c58d4c9 (127.9.174.62:42359): Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
I20260807 08:34:59.798866 19691 leader_election.cc:302] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ea61347409eb4bfc818fa544931f8576; no voters: 5c7b45d28ba243aa8679d3071c58d4c9, 7438c78b2f514c20babb5aae6e8bea9e
I20260807 08:34:59.799085 20500 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 08:35:00.267805 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:00.267075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.1:38583)
0807 08:35:00.267529 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:00.267632 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":169,"threads_started":1}
I20260807 08:35:00.267987 20515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:00.267075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.3:58107)
0807 08:35:00.267767 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:00.267852 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":171,"threads_started":1}
I20260807 08:35:00.268157 20518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:00.267065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.2:54721)
0807 08:35:00.267926 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:00.268014 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":192,"threads_started":1}
I20260807 08:35:01.269476 20519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.268831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.1:45237)
0807 08:35:01.269239 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.269329 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":118,"threads_started":1}
I20260807 08:35:01.269784 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.269219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.2:37895)
0807 08:35:01.269570 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.269646 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":83,"threads_started":1}
I20260807 08:35:01.269829 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.269179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.3:53773)
0807 08:35:01.269655 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.269718 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":98,"threads_started":1}
I20260807 08:35:01.397732 20523 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:35:01.397958 20523 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } }
I20260807 08:35:01.398564 20523 leader_election.cc:288] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c7b45d28ba243aa8679d3071c58d4c9 (127.9.174.62:42359), ea61347409eb4bfc818fa544931f8576 (127.9.174.60:45863)
I20260807 08:35:01.399549 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.398989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:38902)
0807 08:35:01.399348 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.399429 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":87,"threads_started":1}
I20260807 08:35:01.399914 19730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7438c78b2f514c20babb5aae6e8bea9e" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ea61347409eb4bfc818fa544931f8576" is_pre_election: true
I20260807 08:35:01.400132 19730 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7438c78b2f514c20babb5aae6e8bea9e in term 2.
I20260807 08:35:01.400628 19417 leader_election.cc:302] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7438c78b2f514c20babb5aae6e8bea9e, ea61347409eb4bfc818fa544931f8576; no voters:
I20260807 08:35:01.400743 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.400446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:38918)
0807 08:35:01.400578 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.400632 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 08:35:01.400913 20523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 08:35:01.400987 20523 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:35:01.401018 20523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 2 FOLLOWER]: Advancing to term 3
W20260807 08:35:01.401189 19417 leader_election.cc:334] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5c7b45d28ba243aa8679d3071c58d4c9 (127.9.174.62:42359): Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
I20260807 08:35:01.402220 20523 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } }
I20260807 08:35:01.402671 20523 leader_election.cc:288] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [CANDIDATE]: Term 3 election: Requested vote from peers 5c7b45d28ba243aa8679d3071c58d4c9 (127.9.174.62:42359), ea61347409eb4bfc818fa544931f8576 (127.9.174.60:45863)
I20260807 08:35:01.403363 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.403104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:38930)
0807 08:35:01.403256 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.403307 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 08:35:01.403721 19730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7438c78b2f514c20babb5aae6e8bea9e" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ea61347409eb4bfc818fa544931f8576"
I20260807 08:35:01.403954 19730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 2 FOLLOWER]: Advancing to term 3
I20260807 08:35:01.404170 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.403893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:38934)
0807 08:35:01.404058 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.404106 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260807 08:35:01.404285 19417 leader_election.cc:334] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 5c7b45d28ba243aa8679d3071c58d4c9 (127.9.174.62:42359): Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
I20260807 08:35:01.405606 19730 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7438c78b2f514c20babb5aae6e8bea9e in term 3.
I20260807 08:35:01.406033 19417 leader_election.cc:302] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7438c78b2f514c20babb5aae6e8bea9e, ea61347409eb4bfc818fa544931f8576; no voters: 5c7b45d28ba243aa8679d3071c58d4c9
I20260807 08:35:01.406311 20523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 3 FOLLOWER]: Leader election won for term 3
I20260807 08:35:01.406606 20523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [term 3 LEADER]: Becoming Leader. State: Replica: 7438c78b2f514c20babb5aae6e8bea9e, State: Running, Role: LEADER
I20260807 08:35:01.406991 20523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } }
I20260807 08:35:01.408568 20529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7438c78b2f514c20babb5aae6e8bea9e. Latest consensus state: current_term: 3 leader_uuid: "7438c78b2f514c20babb5aae6e8bea9e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } } }
I20260807 08:35:01.408744 20529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: This master's current role is: LEADER
I20260807 08:35:01.409176 20533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:35:01.412305 20533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:35:01.412854 20533 catalog_manager.cc:1259] Loaded cluster ID: 52ca67c8727643308ba3e9a056ffe10c
I20260807 08:35:01.412919 20533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:35:01.413681 20533 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:35:01.414196 20533 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: Loaded TSK: 0
I20260807 08:35:01.415226 20533 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:35:01.443773 19439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:35:01.447436 19439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:35:01.451591 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.451159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:38944)
0807 08:35:01.451361 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.451464 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 08:35:01.451612 19730 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [term 3 FOLLOWER]: Refusing update from remote peer 7438c78b2f514c20babb5aae6e8bea9e: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260807 08:35:01.452194 20529 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 08:35:01.452639 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.452338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.0.0.1:38956)
0807 08:35:01.452504 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.452566 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260807 08:35:01.452805 19417 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e -> Peer 5c7b45d28ba243aa8679d3071c58d4c9 (127.9.174.62:42359): Couldn't send request to peer 5c7b45d28ba243aa8679d3071c58d4c9. Status: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 08:35:01.454118 20536 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7438c78b2f514c20babb5aae6e8bea9e. Latest consensus state: current_term: 3 leader_uuid: "7438c78b2f514c20babb5aae6e8bea9e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } } }
I20260807 08:35:01.454288 20536 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:35:01.455924 20529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "7438c78b2f514c20babb5aae6e8bea9e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } } }
I20260807 08:35:01.456063 20529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e [sys.catalog]: This master's current role is: LEADER
I20260807 08:35:01.456636 20536 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "7438c78b2f514c20babb5aae6e8bea9e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5c7b45d28ba243aa8679d3071c58d4c9" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42359 } } peers { permanent_uuid: "7438c78b2f514c20babb5aae6e8bea9e" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 37673 } attrs { promote: false } } peers { permanent_uuid: "ea61347409eb4bfc818fa544931f8576" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45863 } attrs { promote: false } } }
I20260807 08:35:01.456799 20536 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea61347409eb4bfc818fa544931f8576 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:35:01.473109 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.468261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.9.174.3:36867)
0807 08:35:01.469000 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:01.469004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:01.469024 (+ 20us) server_negotiation.cc:407] Connection header received
0807 08:35:01.469095 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:01.469098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:01.469157 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:01.469263 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:01.469587 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.470365 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.471734 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.472353 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.472469 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:01.472550 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:01.472867 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 08:35:01.472949 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":87,"threads_started":1}
I20260807 08:35:01.473124 20546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.467733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:36867 (local address 127.0.0.1:41700)
0807 08:35:01.468890 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.468907 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:01.469004 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:01.469297 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:01.469300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:01.469321 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:01.469439 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:01.469447 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.470486 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.470492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:01.471600 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:01.471611 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.472629 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.472632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:01.472718 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:01.472741 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:35:01.472846 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 08:35:01.472940 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1094,"mutex_wait_us":97,"thread_start_us":137,"threads_started":1}
I20260807 08:35:01.473541 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.467101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:41967 (local address 127.0.0.1:48266)
0807 08:35:01.467575 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.467592 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:01.467713 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:01.468224 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:01.468228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:01.468247 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:01.468400 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:01.468408 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.470108 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.470112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:01.471820 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:01.471835 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.473010 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.473014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:01.473128 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:01.473168 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 08:35:01.473229 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 08:35:01.473340 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":128}
I20260807 08:35:01.473912 20548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.467592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54204 (local address 127.9.174.2:33389)
0807 08:35:01.467981 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:01.467986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:01.468350 (+ 364us) server_negotiation.cc:407] Connection header received
0807 08:35:01.468441 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:01.468447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:01.468524 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:01.468623 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:01.469149 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.469965 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.472042 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.472811 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.472965 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:01.473058 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:01.473612 (+ 554us) server_negotiation.cc:299] Negotiation successful
0807 08:35:01.473735 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":110,"threads_started":1}
I20260807 08:35:01.473543 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.467446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:33389 (local address 127.0.0.1:54204)
0807 08:35:01.468059 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.468079 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:01.468232 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:01.468783 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:01.468786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:01.468805 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:01.468979 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:01.468988 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.470115 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.470119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:01.471881 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:01.471892 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.473120 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.473124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:01.473249 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:01.473270 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 08:35:01.473338 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 08:35:01.473429 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":179,"thread_start_us":163,"threads_started":1}
I20260807 08:35:01.474318 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.467276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48266 (local address 127.9.174.1:41967)
0807 08:35:01.467921 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:01.467926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:01.467941 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:35:01.468004 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:01.468007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:01.468055 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:01.468524 (+ 469us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:01.468654 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.469958 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.472048 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.472818 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.473838 (+ 1020us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:01.473947 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:01.474001 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 08:35:01.474131 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":123,"threads_started":1}
I20260807 08:35:01.474251 20299 tablet_service.cc:1467] Processing CreateTablet for tablet 898868ca7cea4928884324049ff5d435 (DEFAULT_TABLE table=test-table [id=eebc8f4f72a04a918d955beca190b110]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:01.474679 20299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 898868ca7cea4928884324049ff5d435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:01.474924 20092 tablet_service.cc:1467] Processing CreateTablet for tablet 898868ca7cea4928884324049ff5d435 (DEFAULT_TABLE table=test-table [id=eebc8f4f72a04a918d955beca190b110]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:01.475080 19884 tablet_service.cc:1467] Processing CreateTablet for tablet 898868ca7cea4928884324049ff5d435 (DEFAULT_TABLE table=test-table [id=eebc8f4f72a04a918d955beca190b110]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:01.475365 20092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 898868ca7cea4928884324049ff5d435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:01.475437 19884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 898868ca7cea4928884324049ff5d435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:01.477628 20553 tablet_bootstrap.cc:492] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832: Bootstrap starting.
I20260807 08:35:01.478753 20555 tablet_bootstrap.cc:492] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54: Bootstrap starting.
I20260807 08:35:01.478888 20553 tablet_bootstrap.cc:654] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:01.479914 20555 tablet_bootstrap.cc:654] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:01.480485 20553 tablet_bootstrap.cc:492] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832: No bootstrap required, opened a new log
I20260807 08:35:01.480599 20553 ts_tablet_manager.cc:1397] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:35:01.480842 20554 tablet_bootstrap.cc:492] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4: Bootstrap starting.
I20260807 08:35:01.481158 20553 raft_consensus.cc:348] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } } peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } }
I20260807 08:35:01.481302 20553 raft_consensus.cc:374] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:01.481338 20553 raft_consensus.cc:729] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f945358cf4aa423394dbbe1bbea16832, State: Initialized, Role: FOLLOWER
I20260807 08:35:01.481552 20553 consensus_queue.cc:260] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } } peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } }
I20260807 08:35:01.481976 20554 tablet_bootstrap.cc:654] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:01.482218 20553 ts_tablet_manager.cc:1428] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:35:01.483310 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.482758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.3:58725)
0807 08:35:01.483123 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.483200 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267}
I20260807 08:35:01.484086 20554 tablet_bootstrap.cc:492] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4: No bootstrap required, opened a new log
I20260807 08:35:01.484208 20554 ts_tablet_manager.cc:1397] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:35:01.484216 20555 tablet_bootstrap.cc:492] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54: No bootstrap required, opened a new log
I20260807 08:35:01.484375 20555 ts_tablet_manager.cc:1397] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 08:35:01.484853 20555 raft_consensus.cc:348] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } } peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } }
I20260807 08:35:01.484999 20555 raft_consensus.cc:374] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:01.484896 20554 raft_consensus.cc:348] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } } peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } }
I20260807 08:35:01.485030 20555 raft_consensus.cc:729] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41ee8b27a214c348b0e0b86a2d74f54, State: Initialized, Role: FOLLOWER
I20260807 08:35:01.485046 20554 raft_consensus.cc:374] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:01.485092 20554 raft_consensus.cc:729] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c02c956afe4c4c4f8ff1353b2e92beb4, State: Initialized, Role: FOLLOWER
I20260807 08:35:01.485174 20555 consensus_queue.cc:260] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } } peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } }
I20260807 08:35:01.485255 20554 consensus_queue.cc:260] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } } peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } }
I20260807 08:35:01.485582 20555 ts_tablet_manager.cc:1428] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 08:35:01.485754 20554 ts_tablet_manager.cc:1428] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 08:35:01.486610 20519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.486086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.1:54525)
0807 08:35:01.486452 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.486519 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279}
I20260807 08:35:01.486968 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.486256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.2:35723)
0807 08:35:01.486817 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.486878 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473}
I20260807 08:35:01.487305 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.486837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.3:51871)
0807 08:35:01.487120 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.487181 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198}
I20260807 08:35:01.488018 20401 heartbeater.cc:503] Master 127.9.174.61:37673 was elected leader, sending a full tablet report...
I20260807 08:35:01.490731 20201 heartbeater.cc:503] Master 127.9.174.61:37673 was elected leader, sending a full tablet report...
I20260807 08:35:01.491104 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.490553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.2:44213)
0807 08:35:01.490938 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.491024 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310}
I20260807 08:35:01.492128 19990 heartbeater.cc:503] Master 127.9.174.61:37673 was elected leader, sending a full tablet report...
I20260807 08:35:01.492995 20519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.492535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.1:50419)
0807 08:35:01.492847 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.492906 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225}
W20260807 08:35:01.551231 20402 tablet.cc:2369] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:35:01.586551 20564 raft_consensus.cc:482] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:35:01.586923 20564 raft_consensus.cc:504] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } } peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } }
I20260807 08:35:01.588763 20564 leader_election.cc:288] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c02c956afe4c4c4f8ff1353b2e92beb4 (127.9.174.2:33389), f945358cf4aa423394dbbe1bbea16832 (127.9.174.3:36867)
I20260807 08:35:01.594098 20548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.589554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:35735 (local address 127.9.174.2:33389)
0807 08:35:01.589746 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:01.589751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:01.590094 (+ 343us) server_negotiation.cc:407] Connection header received
0807 08:35:01.590233 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:01.590236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:01.590303 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:01.590382 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:01.590703 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.591390 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.592972 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.593407 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.593545 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:01.593627 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:01.593865 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 08:35:01.593942 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 08:35:01.594187 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.589162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:33389 (local address 127.9.174.1:35735)
0807 08:35:01.589982 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.589995 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:01.590123 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:01.590394 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:01.590397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:01.590417 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:01.590577 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:01.590583 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.591513 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.591516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:01.592845 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:01.592856 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.593578 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.593581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:01.593708 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:01.593743 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 08:35:01.593832 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 08:35:01.593932 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"mutex_wait_us":423,"thread_start_us":187,"threads_started":1}
I20260807 08:35:01.594198 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.589434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:49095 (local address 127.9.174.3:36867)
0807 08:35:01.589655 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:01.589661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:01.589686 (+ 25us) server_negotiation.cc:407] Connection header received
0807 08:35:01.589773 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:01.589777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:01.589848 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:01.590056 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:01.590459 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.591189 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.592995 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.593491 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.593627 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:01.593710 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:01.593928 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 08:35:01.594029 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 08:35:01.594198 20519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.589162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:36867 (local address 127.9.174.1:49095)
0807 08:35:01.589342 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.589361 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:01.589481 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:01.590100 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:01.590105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:01.590134 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:01.590293 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:01.590321 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.591316 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.591321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:01.592837 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:01.592849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.593656 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.593660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:01.593770 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:01.593798 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 08:35:01.593909 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 08:35:01.594020 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":36}
I20260807 08:35:01.595206 20331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898868ca7cea4928884324049ff5d435" candidate_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f945358cf4aa423394dbbe1bbea16832" is_pre_election: true
I20260807 08:35:01.595534 20331 raft_consensus.cc:2454] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f41ee8b27a214c348b0e0b86a2d74f54 in term 0.
I20260807 08:35:01.595897 20098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898868ca7cea4928884324049ff5d435" candidate_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" is_pre_election: true
I20260807 08:35:01.596079 20098 raft_consensus.cc:2454] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f41ee8b27a214c348b0e0b86a2d74f54 in term 0.
I20260807 08:35:01.596161 19813 leader_election.cc:302] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f41ee8b27a214c348b0e0b86a2d74f54, f945358cf4aa423394dbbe1bbea16832; no voters:
I20260807 08:35:01.596570 20564 raft_consensus.cc:2781] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:35:01.596650 20564 raft_consensus.cc:482] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:35:01.596696 20564 raft_consensus.cc:3037] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:01.598116 20564 raft_consensus.cc:504] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } } peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } }
I20260807 08:35:01.598589 20564 leader_election.cc:288] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [CANDIDATE]: Term 1 election: Requested vote from peers c02c956afe4c4c4f8ff1353b2e92beb4 (127.9.174.2:33389), f945358cf4aa423394dbbe1bbea16832 (127.9.174.3:36867)
I20260807 08:35:01.599160 20331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898868ca7cea4928884324049ff5d435" candidate_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f945358cf4aa423394dbbe1bbea16832"
I20260807 08:35:01.599216 20098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898868ca7cea4928884324049ff5d435" candidate_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4"
I20260807 08:35:01.599308 20331 raft_consensus.cc:3037] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:01.599411 20098 raft_consensus.cc:3037] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:01.600606 20331 raft_consensus.cc:2454] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f41ee8b27a214c348b0e0b86a2d74f54 in term 1.
I20260807 08:35:01.600679 20098 raft_consensus.cc:2454] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f41ee8b27a214c348b0e0b86a2d74f54 in term 1.
I20260807 08:35:01.601033 19813 leader_election.cc:302] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f41ee8b27a214c348b0e0b86a2d74f54, f945358cf4aa423394dbbe1bbea16832; no voters:
I20260807 08:35:01.601313 20564 raft_consensus.cc:2781] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:35:01.601666 20564 raft_consensus.cc:686] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [term 1 LEADER]: Becoming Leader. State: Replica: f41ee8b27a214c348b0e0b86a2d74f54, State: Running, Role: LEADER
I20260807 08:35:01.602227 20564 consensus_queue.cc:237] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } } peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } }
I20260807 08:35:01.602597 20519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.602297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.1:54423)
0807 08:35:01.602444 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.602508 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 08:35:01.605695 19438 catalog_manager.cc:5495] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 reported cstate change: term changed from 0 to 1, leader changed from <none> to f41ee8b27a214c348b0e0b86a2d74f54 (127.9.174.1). New cstate: current_term: 1 leader_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41ee8b27a214c348b0e0b86a2d74f54" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 41967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 } health_report { overall_health: UNKNOWN } } }
I20260807 08:35:01.634516 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.629551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48282 (local address 127.9.174.1:41967)
0807 08:35:01.629750 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:01.629754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:01.629777 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:35:01.629931 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:01.629933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:01.630010 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:01.630131 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:35:01.630501 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.631199 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.632487 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:01.632766 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:01.632914 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:01.633312 (+ 398us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:35:01.633950 (+ 638us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:35:01.634059 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:01.634237 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 08:35:01.634315 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 08:35:01.638751 20098 raft_consensus.cc:1261] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 1 FOLLOWER]: Refusing update from remote peer f41ee8b27a214c348b0e0b86a2d74f54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:35:01.639297 20331 raft_consensus.cc:1261] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832 [term 1 FOLLOWER]: Refusing update from remote peer f41ee8b27a214c348b0e0b86a2d74f54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:35:01.639598 20577 consensus_queue.cc:1035] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c02c956afe4c4c4f8ff1353b2e92beb4" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 33389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:35:01.640024 20564 consensus_queue.cc:1035] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f945358cf4aa423394dbbe1bbea16832" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:35:01.642385 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.641989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.2:57001)
0807 08:35:01.642144 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.642263 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 08:35:01.644286 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.643577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.3:42083)
0807 08:35:01.643729 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.643807 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 08:35:01.649713 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.649330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.3:48565)
0807 08:35:01.649537 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.649608 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 08:35:01.649962 20581 mvcc.cc:205] Tried to move back new op lower bound from 7315831609904590848 to 7315831609766825984. Current Snapshot: MvccSnapshot[applied={T|T < 7315831609904590848 or (T in {7315831609904590848})}]
I20260807 08:35:01.650851 20519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.648454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.1:35163)
0807 08:35:01.650689 (+ 2235us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.650757 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2117}
I20260807 08:35:01.651088 20583 mvcc.cc:205] Tried to move back new op lower bound from 7315831609904590848 to 7315831609766825984. Current Snapshot: MvccSnapshot[applied={T|T < 7315831609904590848}]
I20260807 08:35:01.651569 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.650995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42359 (local address 127.9.174.2:48675)
0807 08:35:01.651405 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.651478 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286}
I20260807 08:35:01.661881 19438 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:35:01.662142 19438 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:35:01.668102 19438 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7438c78b2f514c20babb5aae6e8bea9e: Sending DeleteTablet for 3 replicas of tablet 898868ca7cea4928884324049ff5d435
I20260807 08:35:01.669175 19884 tablet_service.cc:1514] Processing DeleteTablet for tablet 898868ca7cea4928884324049ff5d435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:01 UTC) from {username='oryx', principal='oryx/127.9.174.61@KRBTEST.COM'} at 127.0.0.1:48266
I20260807 08:35:01.669502 20092 tablet_service.cc:1514] Processing DeleteTablet for tablet 898868ca7cea4928884324049ff5d435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:01 UTC) from {username='oryx', principal='oryx/127.9.174.61@KRBTEST.COM'} at 127.0.0.1:54204
I20260807 08:35:01.669615 20299 tablet_service.cc:1514] Processing DeleteTablet for tablet 898868ca7cea4928884324049ff5d435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:01 UTC) from {username='oryx', principal='oryx/127.9.174.61@KRBTEST.COM'} at 127.0.0.1:41700
I20260807 08:35:01.670028 20599 tablet_replica.cc:330] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54: stopping tablet replica
I20260807 08:35:01.670197 20599 raft_consensus.cc:2229] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:35:01.670526 20602 tablet_replica.cc:330] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832: stopping tablet replica
I20260807 08:35:01.670753 20599 raft_consensus.cc:2258] T 898868ca7cea4928884324049ff5d435 P f41ee8b27a214c348b0e0b86a2d74f54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:01.670756 20602 raft_consensus.cc:2229] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:35:01.670943 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 19646
I20260807 08:35:01.675034 20600 tablet_replica.cc:330] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4: stopping tablet replica
I20260807 08:35:01.675285 20600 raft_consensus.cc:2229] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:35:01.675513 20600 raft_consensus.cc:2258] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:01.675808 20602 raft_consensus.cc:2258] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:01.678676 20600 ts_tablet_manager.cc:1905] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:35:01.681648 20602 ts_tablet_manager.cc:1905] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:35:01.682348 20600 ts_tablet_manager.cc:1918] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:35:01.682475 20600 log.cc:1196] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/898868ca7cea4928884324049ff5d435
I20260807 08:35:01.682855 20600 ts_tablet_manager.cc:1939] T 898868ca7cea4928884324049ff5d435 P c02c956afe4c4c4f8ff1353b2e92beb4: Deleting consensus metadata
I20260807 08:35:01.684067 19420 catalog_manager.cc:4841] TS c02c956afe4c4c4f8ff1353b2e92beb4 (127.9.174.2:33389): tablet 898868ca7cea4928884324049ff5d435 (table test-table [id=eebc8f4f72a04a918d955beca190b110]) successfully deleted
I20260807 08:35:01.685015 20602 ts_tablet_manager.cc:1918] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:35:01.685156 20602 log.cc:1196] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/898868ca7cea4928884324049ff5d435
I20260807 08:35:01.685475 20602 ts_tablet_manager.cc:1939] T 898868ca7cea4928884324049ff5d435 P f945358cf4aa423394dbbe1bbea16832: Deleting consensus metadata
I20260807 08:35:01.686456 19417 catalog_manager.cc:4841] TS f945358cf4aa423394dbbe1bbea16832 (127.9.174.3:36867): tablet 898868ca7cea4928884324049ff5d435 (table test-table [id=eebc8f4f72a04a918d955beca190b110]) successfully deleted
I20260807 08:35:01.690346 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 19998
I20260807 08:35:01.691232 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:01.690635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:41967 (local address 127.0.0.1:48284)
0807 08:35:01.690899 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:01.690970 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:41967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
W20260807 08:35:01.694474 19416 catalog_manager.cc:4584] TS f41ee8b27a214c348b0e0b86a2d74f54 (127.9.174.1:41967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 898868ca7cea4928884324049ff5d435: Network error: Client connection negotiation failed: client connection to 127.9.174.1:41967: connect: Connection refused (error 111)
I20260807 08:35:01.709936 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 20212
I20260807 08:35:01.730014 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 19238
I20260807 08:35:01.752441 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 19399
2026-08-07T08:35:01Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7973 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:35:01 dist-test-slave-h3ft krb5kdc[20606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:35:01 dist-test-slave-h3ft krb5kdc[20606](info): set up 2 sockets
Aug 07 08:35:01 dist-test-slave-h3ft krb5kdc[20606](info): commencing operation
krb5kdc: starting...
W20260807 08:35:03.078275 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:35:03 dist-test-slave-h3ft krb5kdc[20606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:35:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:35:03Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.62.keytab.
Entry for principal oryx/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.62.keytab.
I20260807 08:35:03.252781 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:39493
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.62.keytab
--principal=oryx/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:46295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:39493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:35:03.397372 20622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:03.397765 20622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:03.397852 20622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:03.402050 20622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:35:03.402165 20622 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 08:35:03.402199 20622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:03.402238 20622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:03.402272 20622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:35:03.402313 20622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:35:03.408461 20622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:46295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:39493
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.62.keytab
--principal=oryx/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:39493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:03.409811 20622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:03.411034 20622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:35:03.422194 20622 server_base.cc:1034] running on GCE node
W20260807 08:35:03.422406 20629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:03.422231 20628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:03.422231 20631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:03.423457 20622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:03.424791 20622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:03.425985 20622 hybrid_clock.cc:648] HybridClock initialized: now 1786091703425947 us; error 66 us; skew 500 ppm
Aug 07 08:35:03 dist-test-slave-h3ft krb5kdc[20606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091703, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:35:03.435796 20622 init.cc:377] Logged in from keytab as oryx/127.9.174.62@KRBTEST.COM (short username oryx)
I20260807 08:35:03.437034 20622 webserver.cc:460] Webserver started at http://127.9.174.62:45571/ using document root <none> and password file <none>
I20260807 08:35:03.437727 20622 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:03.437801 20622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:03.438056 20622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:03.440770 20622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "3471e58ba77f49bfb97c69122ee8d00d"
format_stamp: "Formatted at 2026-08-07 08:35:03 on dist-test-slave-h3ft"
server_key: "25c5a6899f0a13e615e9c40972c1f784"
server_key_iv: "f8607bd9e8c374b67f1e8471d3a71552"
server_key_version: "encryptionkey@0"
I20260807 08:35:03.441432 20622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "3471e58ba77f49bfb97c69122ee8d00d"
format_stamp: "Formatted at 2026-08-07 08:35:03 on dist-test-slave-h3ft"
server_key: "25c5a6899f0a13e615e9c40972c1f784"
server_key_iv: "f8607bd9e8c374b67f1e8471d3a71552"
server_key_version: "encryptionkey@0"
I20260807 08:35:03.446130 20622 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 08:35:03.449374 20640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:03.451160 20622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:35:03.451344 20622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "3471e58ba77f49bfb97c69122ee8d00d"
format_stamp: "Formatted at 2026-08-07 08:35:03 on dist-test-slave-h3ft"
server_key: "25c5a6899f0a13e615e9c40972c1f784"
server_key_iv: "f8607bd9e8c374b67f1e8471d3a71552"
server_key_version: "encryptionkey@0"
I20260807 08:35:03.451480 20622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:03.471266 20622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:03.475939 20622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:03.476182 20622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:03.486263 20622 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:39493
I20260807 08:35:03.486337 20731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:39493 every 8 connection(s)
I20260807 08:35:03.487475 20622 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:35:03.491554 20732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:03.495249 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 20622
I20260807 08:35:03.495443 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:35:03.495783 9912 external_mini_cluster.cc:1428] Setting key 0fef8ca3b52039cc3fc3ee2358ebddae
I20260807 08:35:03.499960 20732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d: Bootstrap starting.
I20260807 08:35:03.503475 20732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:03.504562 20732 log.cc:824] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:35:03 dist-test-slave-h3ft krb5kdc[20606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM
I20260807 08:35:03.507575 20732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d: No bootstrap required, opened a new log
I20260807 08:35:03.512744 20732 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3471e58ba77f49bfb97c69122ee8d00d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39493 } }
I20260807 08:35:03.513094 20732 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:03.513180 20732 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3471e58ba77f49bfb97c69122ee8d00d, State: Initialized, Role: FOLLOWER
I20260807 08:35:03.513731 20732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3471e58ba77f49bfb97c69122ee8d00d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39493 } }
I20260807 08:35:03.513938 20732 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:35:03.514035 20732 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:35:03.514170 20732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:03.514569 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:03.497685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38720 (local address 127.9.174.62:39493)
0807 08:35:03.498338 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:03.498364 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:03.498424 (+ 60us) server_negotiation.cc:407] Connection header received
0807 08:35:03.499410 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:03.499440 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:03.499972 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:03.500425 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:03.501250 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:03.502872 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:03.504023 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:03.504782 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:03.504970 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:03.507881 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:03.507888 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:03.507916 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:03.507955 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:03.510575 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:03.511241 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:03.511251 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:03.511259 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:03.511471 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:03.511827 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:03.511829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:03.511831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:03.512398 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:03.512689 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:03.512965 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 08:35:03.513373 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":167,"threads_started":1}
I20260807 08:35:03.515698 20732 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3471e58ba77f49bfb97c69122ee8d00d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39493 } }
I20260807 08:35:03.516139 20732 leader_election.cc:302] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3471e58ba77f49bfb97c69122ee8d00d; no voters:
I20260807 08:35:03.516618 20732 leader_election.cc:288] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:35:03.517050 20741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:35:03.517488 20741 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [term 1 LEADER]: Becoming Leader. State: Replica: 3471e58ba77f49bfb97c69122ee8d00d, State: Running, Role: LEADER
I20260807 08:35:03.517990 20741 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3471e58ba77f49bfb97c69122ee8d00d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39493 } }
I20260807 08:35:03.518316 20732 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:35:03.520695 20743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3471e58ba77f49bfb97c69122ee8d00d. Latest consensus state: current_term: 1 leader_uuid: "3471e58ba77f49bfb97c69122ee8d00d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3471e58ba77f49bfb97c69122ee8d00d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39493 } } }
I20260807 08:35:03.520876 20743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [sys.catalog]: This master's current role is: LEADER
I20260807 08:35:03.521330 20742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3471e58ba77f49bfb97c69122ee8d00d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3471e58ba77f49bfb97c69122ee8d00d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39493 } } }
I20260807 08:35:03.521504 20742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d [sys.catalog]: This master's current role is: LEADER
W20260807 08:35:03.524992 20761 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:35:03.525098 20761 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:35:03.525188 20762 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:35:03.526070 20762 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:35:03.534075 20762 catalog_manager.cc:1347] Generated new cluster ID: 7447d40b48494157bf2a1d84e044335d
I20260807 08:35:03.534209 20762 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:35:03.546412 20762 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:35:03.547966 20762 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:35:03.558471 20762 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d: Generated new TSK 0
I20260807 08:35:03.559607 20762 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.1.keytab.
Entry for principal oryx/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.1.keytab.
I20260807 08:35:03.612084 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:39493
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.1.keytab
--principal=oryx/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:46295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:35:03.759469 20773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:03.759940 20773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:03.760042 20773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:03.764657 20773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:03.764807 20773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:03.764911 20773 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:35:03.765013 20773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:35:03.771487 20773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:46295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.1.keytab
--principal=oryx/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.174.62:39493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:03.773216 20773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:03.774662 20773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:03.787757 20784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:03.787757 20782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:03.787760 20780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:03.788342 20773 server_base.cc:1034] running on GCE node
I20260807 08:35:03.788889 20773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:03.789696 20773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:03.790989 20773 hybrid_clock.cc:648] HybridClock initialized: now 1786091703790965 us; error 88 us; skew 500 ppm
Aug 07 08:35:03 dist-test-slave-h3ft krb5kdc[20606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091703, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:35:03.795101 20773 init.cc:377] Logged in from keytab as oryx/127.9.174.1@KRBTEST.COM (short username oryx)
I20260807 08:35:03.796495 20773 webserver.cc:460] Webserver started at http://127.9.174.1:41473/ using document root <none> and password file <none>
I20260807 08:35:03.797184 20773 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:03.797268 20773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:03.797466 20773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:03.799847 20773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "39f50f5b25d34bd188b4c6dced28aceb"
format_stamp: "Formatted at 2026-08-07 08:35:03 on dist-test-slave-h3ft"
server_key: "94f38aaa16f697bb28b2d775a0f646e6"
server_key_iv: "098059009ca1ccd7b4331e50cf3a9f23"
server_key_version: "encryptionkey@0"
I20260807 08:35:03.800443 20773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "39f50f5b25d34bd188b4c6dced28aceb"
format_stamp: "Formatted at 2026-08-07 08:35:03 on dist-test-slave-h3ft"
server_key: "94f38aaa16f697bb28b2d775a0f646e6"
server_key_iv: "098059009ca1ccd7b4331e50cf3a9f23"
server_key_version: "encryptionkey@0"
I20260807 08:35:03.806157 20773 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:35:03.810019 20794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:03.812284 20773 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 08:35:03.812523 20773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "39f50f5b25d34bd188b4c6dced28aceb"
format_stamp: "Formatted at 2026-08-07 08:35:03 on dist-test-slave-h3ft"
server_key: "94f38aaa16f697bb28b2d775a0f646e6"
server_key_iv: "098059009ca1ccd7b4331e50cf3a9f23"
server_key_version: "encryptionkey@0"
I20260807 08:35:03.812722 20773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:03.823639 20773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:03.827478 20773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:03.827724 20773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:03.829824 20773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:03.829895 20773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:03.829942 20773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:03.829959 20773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:03.843338 20773 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:33173
I20260807 08:35:03.843461 20955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:33173 every 8 connection(s)
Aug 07 08:35:03 dist-test-slave-h3ft krb5kdc[20606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091703, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM
I20260807 08:35:03.844803 20773 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:35:03.849779 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:03.831674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:55813 (local address 127.9.174.62:39493)
0807 08:35:03.831909 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:03.831914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:03.833056 (+ 1142us) server_negotiation.cc:407] Connection header received
0807 08:35:03.835134 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:03.835143 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:03.835232 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:03.835382 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:03.837421 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:03.838327 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:03.840257 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:03.840685 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:03.840875 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:03.844798 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:03.844803 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:03.844805 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:03.844844 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:03.846806 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:03.847522 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:03.847525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:03.847527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:03.847621 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:03.848016 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:03.848019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:03.848021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:03.848309 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:03.848419 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:03.849365 (+ 946us) server_negotiation.cc:299] Negotiation successful
0807 08:35:03.849517 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 08:35:03.850736 20807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:03.832173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:39493 (local address 127.9.174.1:55813)
0807 08:35:03.832834 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:03.832897 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:03.834712 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:03.835759 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:03.835770 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:03.836489 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:03.837215 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:03.837235 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:03.838530 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:03.838536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:03.840063 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:03.840082 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:03.840944 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:03.840950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:03.841268 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:03.842212 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:03.842235 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:03.844496 (+ 2261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:03.846965 (+ 2469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:03.846971 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:03.846988 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:03.847392 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:03.847741 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:03.847744 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:03.847746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:03.847903 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:03.848436 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:03.848443 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:03.848904 (+ 461us) client_negotiation.cc:769] Sending connection context
0807 08:35:03.849294 (+ 390us) client_negotiation.cc:240] Negotiation successful
0807 08:35:03.849698 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":206,"threads_started":1}
I20260807 08:35:03.852815 20958 heartbeater.cc:347] Connected to a master server at 127.9.174.62:39493
I20260807 08:35:03.853231 20958 heartbeater.cc:464] Registering TS with master...
I20260807 08:35:03.854142 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 20773
I20260807 08:35:03.854343 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:35:03.854354 20958 heartbeater.cc:511] Master 127.9.174.62:39493 requested a full tablet report, sending...
I20260807 08:35:03.854789 9912 external_mini_cluster.cc:1428] Setting key bed9a0803cdcbd910298fd5f8adc6ccc
I20260807 08:35:03.857091 20658 ts_manager.cc:194] Registered new tserver with Master: 39f50f5b25d34bd188b4c6dced28aceb (127.9.174.1:33173)
I20260807 08:35:03.858803 20658 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:55813
I20260807 08:35:03.867632 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:03.856884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:42669 (local address 127.9.174.62:39493)
0807 08:35:03.857233 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:03.857238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:03.857528 (+ 290us) server_negotiation.cc:407] Connection header received
0807 08:35:03.857726 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:03.857730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:03.857774 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:03.857827 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:03.858560 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:03.859295 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:03.860504 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:03.860881 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:03.860965 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:03.862884 (+ 1919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:03.862889 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:03.862892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:03.862934 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:03.865219 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:03.865899 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:03.865902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:03.865904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:03.866074 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:03.866453 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:03.866456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:03.866457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:03.866741 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:03.866837 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:03.867309 (+ 472us) server_negotiation.cc:299] Negotiation successful
0807 08:35:03.867461 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228}
I20260807 08:35:03.867702 20968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:03.856762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:39493 (local address 127.9.174.1:42669)
0807 08:35:03.857438 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:03.857454 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:03.857604 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:03.857986 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:03.857988 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:03.858275 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:03.858426 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:03.858433 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:03.859462 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:03.859465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:03.860340 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:03.860356 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:03.861157 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:03.861163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:03.861300 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:03.862134 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:03.862152 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:03.862607 (+ 455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:03.865406 (+ 2799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:03.865411 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:03.865413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:03.865748 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:03.866199 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:03.866205 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:03.866210 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:03.866328 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:03.866863 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:03.866868 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:03.867081 (+ 213us) client_negotiation.cc:769] Sending connection context
0807 08:35:03.867232 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 08:35:03.867420 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":176,"threads_started":1}
WARNING: no policy specified for oryx/127.9.174.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.174.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.2.keytab.
Entry for principal oryx/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.2.keytab.
I20260807 08:35:03.897500 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:39493
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.2.keytab
--principal=oryx/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:46295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:35:04.028959 20974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:04.029277 20974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:04.029330 20974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:04.034019 20974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:04.034118 20974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:04.034168 20974 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:35:04.034204 20974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:35:04.040093 20974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:46295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.2.keytab
--principal=oryx/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.174.62:39493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:04.041201 20974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:04.042260 20974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:04.053695 20983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:04.053756 20980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:04.053951 20979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:04.054168 20974 server_base.cc:1034] running on GCE node
I20260807 08:35:04.054632 20974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:04.055490 20974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:04.056703 20974 hybrid_clock.cc:648] HybridClock initialized: now 1786091704056686 us; error 42 us; skew 500 ppm
Aug 07 08:35:04 dist-test-slave-h3ft krb5kdc[20606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091704, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:35:04.060022 20974 init.cc:377] Logged in from keytab as oryx/127.9.174.2@KRBTEST.COM (short username oryx)
I20260807 08:35:04.061307 20974 webserver.cc:460] Webserver started at http://127.9.174.2:40765/ using document root <none> and password file <none>
I20260807 08:35:04.062249 20974 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:04.062342 20974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:04.062552 20974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:04.064976 20974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "a8ff8eef27064232b25342734abbc47c"
format_stamp: "Formatted at 2026-08-07 08:35:04 on dist-test-slave-h3ft"
server_key: "807cdba417b69df37c2104f922b742bb"
server_key_iv: "00226468c2195a0475274d8369b46574"
server_key_version: "encryptionkey@0"
I20260807 08:35:04.065564 20974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "a8ff8eef27064232b25342734abbc47c"
format_stamp: "Formatted at 2026-08-07 08:35:04 on dist-test-slave-h3ft"
server_key: "807cdba417b69df37c2104f922b742bb"
server_key_iv: "00226468c2195a0475274d8369b46574"
server_key_version: "encryptionkey@0"
I20260807 08:35:04.070376 20974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260807 08:35:04.074295 20993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:04.076220 20974 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 08:35:04.076520 20974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "a8ff8eef27064232b25342734abbc47c"
format_stamp: "Formatted at 2026-08-07 08:35:04 on dist-test-slave-h3ft"
server_key: "807cdba417b69df37c2104f922b742bb"
server_key_iv: "00226468c2195a0475274d8369b46574"
server_key_version: "encryptionkey@0"
I20260807 08:35:04.076769 20974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:04.092833 20974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:04.097160 20974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:04.097429 20974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:04.099589 20974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:04.099674 20974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:04.099763 20974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:04.099788 20974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:35:04 dist-test-slave-h3ft krb5kdc[20606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091704, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM
I20260807 08:35:04.114224 20974 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:44939
I20260807 08:35:04.114329 21153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:44939 every 8 connection(s)
I20260807 08:35:04.115549 20974 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:35:04.117580 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 20974
I20260807 08:35:04.117693 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:35:04.118074 9912 external_mini_cluster.cc:1428] Setting key aa56f18e3d9cb7d9560b2ed3089d6891
I20260807 08:35:04.121054 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.100914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:56649 (local address 127.9.174.62:39493)
0807 08:35:04.101088 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:04.101092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:04.102472 (+ 1380us) server_negotiation.cc:407] Connection header received
0807 08:35:04.104170 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:04.104174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:04.104255 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:04.104372 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:04.106475 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.107502 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.109574 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.109856 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.109983 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.114876 (+ 4893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:04.114881 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:04.114884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:04.114922 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:04.117608 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:04.118433 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:04.118439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:04.118441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:04.118555 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:04.119227 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:04.119230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:04.119233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:04.119541 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:04.119652 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:04.120627 (+ 975us) server_negotiation.cc:299] Negotiation successful
0807 08:35:04.120779 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 08:35:04.122267 21006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.101507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:39493 (local address 127.9.174.2:56649)
0807 08:35:04.102207 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:04.102271 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:04.103876 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:04.104640 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:04.104655 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:04.105305 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:04.106185 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.106225 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.107713 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.107717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.109380 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.109393 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.110079 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.110084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.110381 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.111467 (+ 1086us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:04.111496 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:04.114585 (+ 3089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:04.117836 (+ 3251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:04.117843 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:04.117861 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:04.118307 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:04.118719 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:04.118722 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:04.118724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:04.119094 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:04.119862 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:04.119871 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:04.120316 (+ 445us) client_negotiation.cc:769] Sending connection context
0807 08:35:04.120551 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 08:35:04.120997 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":168,"threads_started":1}
I20260807 08:35:04.123778 21154 heartbeater.cc:347] Connected to a master server at 127.9.174.62:39493
I20260807 08:35:04.123996 21154 heartbeater.cc:464] Registering TS with master...
I20260807 08:35:04.124593 21154 heartbeater.cc:511] Master 127.9.174.62:39493 requested a full tablet report, sending...
I20260807 08:35:04.125936 20658 ts_manager.cc:194] Registered new tserver with Master: a8ff8eef27064232b25342734abbc47c (127.9.174.2:44939)
I20260807 08:35:04.126972 20658 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:56649
WARNING: no policy specified for oryx/127.9.174.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.174.3@KRBTEST.COM" created.
I20260807 08:35:04.138535 21162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.127432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:39493 (local address 127.9.174.2:56293)
0807 08:35:04.128111 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:04.128134 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:04.128322 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:04.128704 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:04.128709 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:04.129047 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:04.129322 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.129336 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.130485 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.130492 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.131637 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.131652 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.132185 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.132189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.132318 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.133058 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:04.133080 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:04.133644 (+ 564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:04.136517 (+ 2873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:04.136520 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:04.136522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:04.136793 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:04.137121 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:04.137123 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:04.137125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:04.137254 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:04.137841 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:04.137847 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:04.137999 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 08:35:04.138138 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 08:35:04.138331 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":4,"thread_start_us":157,"threads_started":1}
I20260807 08:35:04.138643 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.127735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:56293 (local address 127.9.174.62:39493)
0807 08:35:04.127929 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:04.127936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:04.128274 (+ 338us) server_negotiation.cc:407] Connection header received
0807 08:35:04.128459 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:04.128463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:04.128525 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:04.128645 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:04.129507 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.130287 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.131788 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.132017 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.132122 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.133962 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:04.133972 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:04.133976 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:04.134018 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:04.136323 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:04.136911 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:04.136913 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:04.136915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:04.137018 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:04.137364 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:04.137370 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:04.137372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:04.137728 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:04.137854 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:04.138244 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 08:35:04.138401 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.3.keytab.
Entry for principal oryx/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.3.keytab.
I20260807 08:35:04.160023 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:39493
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.3.keytab
--principal=oryx/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:46295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:35:04.303341 21164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:04.303795 21164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:04.303894 21164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:04.308408 21164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:04.308559 21164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:04.308638 21164 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:35:04.308741 21164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:35:04.315128 21164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:46295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.3.keytab
--principal=oryx/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.174.62:39493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:04.316887 21164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:04.318501 21164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:35:04.333469 21164 server_base.cc:1034] running on GCE node
W20260807 08:35:04.333323 21176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:04.333353 21173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:04.333357 21174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:04.334477 21164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:04.335554 21164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:04.336846 21164 hybrid_clock.cc:648] HybridClock initialized: now 1786091704336795 us; error 71 us; skew 500 ppm
Aug 07 08:35:04 dist-test-slave-h3ft krb5kdc[20606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091704, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:35:04.341089 21164 init.cc:377] Logged in from keytab as oryx/127.9.174.3@KRBTEST.COM (short username oryx)
I20260807 08:35:04.342640 21164 webserver.cc:460] Webserver started at http://127.9.174.3:46581/ using document root <none> and password file <none>
I20260807 08:35:04.343559 21164 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:04.343650 21164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:04.343907 21164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:04.346756 21164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "b7e22386798041d695b3f4cbcd82f494"
format_stamp: "Formatted at 2026-08-07 08:35:04 on dist-test-slave-h3ft"
server_key: "c90b6dbfbcebedf76cfdc4c108d3ed56"
server_key_iv: "98ac117b3d29b0cb7632491782c97828"
server_key_version: "encryptionkey@0"
I20260807 08:35:04.347605 21164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "b7e22386798041d695b3f4cbcd82f494"
format_stamp: "Formatted at 2026-08-07 08:35:04 on dist-test-slave-h3ft"
server_key: "c90b6dbfbcebedf76cfdc4c108d3ed56"
server_key_iv: "98ac117b3d29b0cb7632491782c97828"
server_key_version: "encryptionkey@0"
I20260807 08:35:04.353436 21164 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 08:35:04.357568 21187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:04.360219 21164 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260807 08:35:04.360458 21164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "b7e22386798041d695b3f4cbcd82f494"
format_stamp: "Formatted at 2026-08-07 08:35:04 on dist-test-slave-h3ft"
server_key: "c90b6dbfbcebedf76cfdc4c108d3ed56"
server_key_iv: "98ac117b3d29b0cb7632491782c97828"
server_key_version: "encryptionkey@0"
I20260807 08:35:04.360651 21164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:04.396449 21164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:04.400846 21164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:04.401141 21164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:04.403218 21164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:04.403285 21164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:04.403371 21164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:04.403412 21164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:35:04 dist-test-slave-h3ft krb5kdc[20606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091704, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM
I20260807 08:35:04.418198 21164 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:36969
I20260807 08:35:04.418845 21362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:36969 every 8 connection(s)
I20260807 08:35:04.419833 21164 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:35:04.421749 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 21164
I20260807 08:35:04.421931 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:35:04.422360 9912 external_mini_cluster.cc:1428] Setting key e321479596c1c7dd46d7eeeb22f9c77c
I20260807 08:35:04.422544 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.405090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:57719 (local address 127.9.174.62:39493)
0807 08:35:04.405247 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:04.405252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:04.407042 (+ 1790us) server_negotiation.cc:407] Connection header received
0807 08:35:04.408693 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:04.408698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:04.408789 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:04.408949 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:04.411179 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.412096 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.413497 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.413777 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.413900 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.417090 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:04.417095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:04.417099 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:04.417147 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:04.419369 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:04.420312 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:04.420338 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:04.420343 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:04.420482 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:04.421050 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:04.421053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:04.421054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:04.421257 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:04.421344 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:04.422254 (+ 910us) server_negotiation.cc:299] Negotiation successful
0807 08:35:04.422409 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 08:35:04.423882 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.405804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:39493 (local address 127.9.174.3:57719)
0807 08:35:04.406713 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:04.406865 (+ 152us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:04.408292 (+ 1427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:04.409275 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:04.409288 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:04.410056 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:04.410963 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.410988 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.412245 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.412248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.413349 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.413360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.413973 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.413978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.414184 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.414931 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:04.414952 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:04.416770 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:04.419569 (+ 2799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:04.419576 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:04.419599 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:04.420154 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:04.420624 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:04.420628 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:04.420630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:04.420901 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:04.421387 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:04.421394 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:04.421849 (+ 455us) client_negotiation.cc:769] Sending connection context
0807 08:35:04.422146 (+ 297us) client_negotiation.cc:240] Negotiation successful
0807 08:35:04.422618 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":197,"threads_started":1}
I20260807 08:35:04.425890 21363 heartbeater.cc:347] Connected to a master server at 127.9.174.62:39493
I20260807 08:35:04.426208 21363 heartbeater.cc:464] Registering TS with master...
I20260807 08:35:04.427165 21363 heartbeater.cc:511] Master 127.9.174.62:39493 requested a full tablet report, sending...
I20260807 08:35:04.428943 20658 ts_manager.cc:194] Registered new tserver with Master: b7e22386798041d695b3f4cbcd82f494 (127.9.174.3:36969)
I20260807 08:35:04.429966 20658 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:57719
I20260807 08:35:04.437711 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:35:04.438760 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.428620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:39493 (local address 127.9.174.3:41479)
0807 08:35:04.429140 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:04.429166 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:04.429339 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:04.429684 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:04.429688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:04.430044 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:04.430221 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.430231 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.431561 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.431564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.432320 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.432332 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.433059 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.433062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.433155 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.433810 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:04.433822 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:04.434235 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:04.436520 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:04.436523 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:04.436526 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:04.436893 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:04.437261 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:04.437264 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:04.437265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:04.437423 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:04.438071 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:04.438075 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:04.438208 (+ 133us) client_negotiation.cc:769] Sending connection context
0807 08:35:04.438328 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 08:35:04.438551 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":135,"threads_started":1}
I20260807 08:35:04.438838 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.428693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:41479 (local address 127.9.174.62:39493)
0807 08:35:04.428845 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:04.428849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:04.429345 (+ 496us) server_negotiation.cc:407] Connection header received
0807 08:35:04.429452 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:04.429455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:04.429520 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:04.429622 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:04.430432 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.431242 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.432491 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.432876 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.433027 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.434436 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:04.434439 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:04.434441 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:04.434470 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:04.436372 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:04.437016 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:04.437020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:04.437021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:04.437129 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:04.437557 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:04.437560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:04.437562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:04.437929 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:04.438044 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:04.438440 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 08:35:04.438612 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 08:35:04 dist-test-slave-h3ft krb5kdc[20606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091704, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:35:04 dist-test-slave-h3ft krb5kdc[20606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:04 dist-test-slave-h3ft krb5kdc[20606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 08:35:04.470849 20735 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:04.462965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38722 (local address 127.9.174.62:39493)
0807 08:35:04.463197 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:04.463201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:04.463214 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:35:04.463266 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:04.463268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:04.463315 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:04.463391 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:04.464546 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.465130 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.466272 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.466545 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.466692 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.470705 (+ 4013us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38722: BlockingRecv error: recv got EOF from 127.0.0.1:38722 (error 108)
Metrics: {"server-negotiator.queue_time_us":137}
Aug 07 08:35:04 dist-test-slave-h3ft krb5kdc[20606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091704, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM
I20260807 08:35:04.483566 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.473112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38724 (local address 127.9.174.62:39493)
0807 08:35:04.473291 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:04.473294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:04.473367 (+ 73us) server_negotiation.cc:407] Connection header received
0807 08:35:04.473533 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:04.473536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:04.473583 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:04.473669 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:04.474488 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.475131 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.476051 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.476266 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.476364 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.479534 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:04.479538 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:04.479540 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:04.479573 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:04.481442 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:04.481981 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:04.481984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:04.481985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:04.482084 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:04.482489 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:04.482492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:04.482493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:04.482729 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:04.482862 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:04.483258 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 08:35:04.483392 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 08:35:04.488495 20658 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:35:04.492158 20658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:35:04.511778 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.504817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49690 (local address 127.9.174.2:44939)
0807 08:35:04.505164 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:04.505177 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:04.505735 (+ 558us) server_negotiation.cc:407] Connection header received
0807 08:35:04.505858 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:04.505865 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:04.506138 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:04.506330 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:04.506757 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.507843 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.509646 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.510571 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.510699 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.510927 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:04.511417 (+ 490us) server_negotiation.cc:299] Negotiation successful
0807 08:35:04.511569 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":135,"threads_started":1}
I20260807 08:35:04.511956 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.503662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:36969 (local address 127.0.0.1:56768)
0807 08:35:04.504110 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:04.504172 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:04.504326 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:04.506116 (+ 1790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:04.506120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:04.506167 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:04.506414 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.506426 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.508041 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.508044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.509900 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.509907 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.511249 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.511253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.511412 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.511440 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 08:35:04.511522 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:35:04.511753 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":216,"spinlock_wait_cycles":10624,"thread_start_us":129,"threads_started":1}
I20260807 08:35:04.511984 21406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.505162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41762 (local address 127.9.174.1:33173)
0807 08:35:04.505532 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:04.505541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:04.505570 (+ 29us) server_negotiation.cc:407] Connection header received
0807 08:35:04.505666 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:04.505672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:04.505914 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:04.506105 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:04.506641 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.507900 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.510092 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.511117 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.511241 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.511432 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:04.511629 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 08:35:04.511756 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":107,"threads_started":1}
I20260807 08:35:04.512307 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.503662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:33173 (local address 127.0.0.1:41762)
0807 08:35:04.504626 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:04.504643 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:04.504764 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:04.506110 (+ 1346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:04.506118 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:04.506165 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:04.506427 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.506436 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.508268 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.508271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.509883 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.509899 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.511263 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.511267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.511444 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.511472 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 08:35:04.511555 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 08:35:04.512155 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":508,"spinlock_wait_cycles":8832,"thread_start_us":149,"threads_started":1}
I20260807 08:35:04.512606 21401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.504619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:44939 (local address 127.0.0.1:49690)
0807 08:35:04.505571 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:04.505584 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:04.505729 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:04.506508 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:04.506511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:04.506532 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:04.506619 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.506625 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.508020 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.508027 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.509424 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.509440 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.511022 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.511027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.511146 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.511213 (+ 67us) client_negotiation.cc:769] Sending connection context
0807 08:35:04.512264 (+ 1051us) client_negotiation.cc:240] Negotiation successful
0807 08:35:04.512356 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":881,"mutex_wait_us":198,"thread_start_us":115,"threads_started":1}
I20260807 08:35:04.512802 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.505024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56768 (local address 127.9.174.3:36969)
0807 08:35:04.505531 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:04.505543 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:04.505570 (+ 27us) server_negotiation.cc:407] Connection header received
0807 08:35:04.505670 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:04.505675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:04.505914 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:04.506596 (+ 682us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:04.506768 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.507919 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.510085 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.511084 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.511692 (+ 608us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.512004 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:04.512151 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 08:35:04.512270 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":171,"threads_started":1}
I20260807 08:35:04.516117 20859 tablet_service.cc:1467] Processing CreateTablet for tablet 993bd5d07345421b9ea03dd7f31222a6 (DEFAULT_TABLE table=test-table [id=3361acb75dc44eddacbfa2af7fd835db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:04.516605 21060 tablet_service.cc:1467] Processing CreateTablet for tablet 993bd5d07345421b9ea03dd7f31222a6 (DEFAULT_TABLE table=test-table [id=3361acb75dc44eddacbfa2af7fd835db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:04.517423 20859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 993bd5d07345421b9ea03dd7f31222a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:04.517051 21264 tablet_service.cc:1467] Processing CreateTablet for tablet 993bd5d07345421b9ea03dd7f31222a6 (DEFAULT_TABLE table=test-table [id=3361acb75dc44eddacbfa2af7fd835db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:04.517944 21060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 993bd5d07345421b9ea03dd7f31222a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:04.517971 21264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 993bd5d07345421b9ea03dd7f31222a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:04.526899 21407 tablet_bootstrap.cc:492] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb: Bootstrap starting.
I20260807 08:35:04.528059 21409 tablet_bootstrap.cc:492] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494: Bootstrap starting.
I20260807 08:35:04.528617 21408 tablet_bootstrap.cc:492] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c: Bootstrap starting.
I20260807 08:35:04.530463 21407 tablet_bootstrap.cc:654] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:04.531222 21408 tablet_bootstrap.cc:654] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:04.531358 21409 tablet_bootstrap.cc:654] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:04.531576 21407 log.cc:824] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb: Log is configured to *not* fsync() on all Append() calls
I20260807 08:35:04.532089 21408 log.cc:824] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c: Log is configured to *not* fsync() on all Append() calls
I20260807 08:35:04.532439 21409 log.cc:824] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494: Log is configured to *not* fsync() on all Append() calls
I20260807 08:35:04.534560 21407 tablet_bootstrap.cc:492] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb: No bootstrap required, opened a new log
I20260807 08:35:04.534792 21408 tablet_bootstrap.cc:492] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c: No bootstrap required, opened a new log
I20260807 08:35:04.534847 21407 ts_tablet_manager.cc:1397] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260807 08:35:04.534991 21408 ts_tablet_manager.cc:1397] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260807 08:35:04.536852 21409 tablet_bootstrap.cc:492] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494: No bootstrap required, opened a new log
I20260807 08:35:04.537312 21409 ts_tablet_manager.cc:1397] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 08:35:04.540052 21408 raft_consensus.cc:348] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } }
I20260807 08:35:04.540249 21407 raft_consensus.cc:348] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } }
I20260807 08:35:04.540470 21408 raft_consensus.cc:374] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:04.540479 21407 raft_consensus.cc:374] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:04.540544 21408 raft_consensus.cc:729] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ff8eef27064232b25342734abbc47c, State: Initialized, Role: FOLLOWER
I20260807 08:35:04.540544 21407 raft_consensus.cc:729] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f50f5b25d34bd188b4c6dced28aceb, State: Initialized, Role: FOLLOWER
I20260807 08:35:04.541153 21408 consensus_queue.cc:260] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } }
I20260807 08:35:04.541153 21407 consensus_queue.cc:260] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } }
I20260807 08:35:04.541059 21409 raft_consensus.cc:348] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } }
I20260807 08:35:04.541435 21409 raft_consensus.cc:374] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:04.541498 21409 raft_consensus.cc:729] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e22386798041d695b3f4cbcd82f494, State: Initialized, Role: FOLLOWER
I20260807 08:35:04.542060 21409 consensus_queue.cc:260] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } }
I20260807 08:35:04.542495 21408 ts_tablet_manager.cc:1428] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260807 08:35:04.542495 21407 ts_tablet_manager.cc:1428] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb: Time spent starting tablet: real 0.008s user 0.008s sys 0.001s
I20260807 08:35:04.542989 20958 heartbeater.cc:503] Master 127.9.174.62:39493 was elected leader, sending a full tablet report...
I20260807 08:35:04.543115 21154 heartbeater.cc:503] Master 127.9.174.62:39493 was elected leader, sending a full tablet report...
I20260807 08:35:04.543390 21409 ts_tablet_manager.cc:1428] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 08:35:04.543733 21363 heartbeater.cc:503] Master 127.9.174.62:39493 was elected leader, sending a full tablet report...
W20260807 08:35:04.596902 20959 tablet.cc:2369] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:35:04.604696 21414 raft_consensus.cc:482] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:35:04.605146 21414 raft_consensus.cc:504] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } }
I20260807 08:35:04.606742 21414 leader_election.cc:288] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7e22386798041d695b3f4cbcd82f494 (127.9.174.3:36969), a8ff8eef27064232b25342734abbc47c (127.9.174.2:44939)
I20260807 08:35:04.611624 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.607467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:35437 (local address 127.9.174.3:36969)
0807 08:35:04.607661 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:04.607665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:04.607826 (+ 161us) server_negotiation.cc:407] Connection header received
0807 08:35:04.608011 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:04.608015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:04.608090 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:04.608221 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:04.608620 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.609203 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.610491 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.610930 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.611095 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.611180 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:04.611379 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 08:35:04.611463 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 08:35:04.611660 21418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.607227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:36969 (local address 127.9.174.1:35437)
0807 08:35:04.607690 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:04.607709 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:04.607853 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:04.608283 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:04.608287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:04.608330 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:04.608481 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.608488 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.609337 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.609343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.610341 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.610351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.611098 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.611102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.611211 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.611244 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 08:35:04.611339 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 08:35:04.611437 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":3,"thread_start_us":212,"threads_started":1}
I20260807 08:35:04.612267 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.608121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:39549 (local address 127.9.174.2:44939)
0807 08:35:04.608330 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:04.608338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:04.608377 (+ 39us) server_negotiation.cc:407] Connection header received
0807 08:35:04.608444 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:04.608446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:04.608494 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:04.608586 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:04.608900 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.609436 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.610850 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.611392 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.611530 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.611609 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:04.611973 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 08:35:04.612122 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 08:35:04.612262 21419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.607820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:44939 (local address 127.9.174.1:39549)
0807 08:35:04.608143 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:04.608154 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:04.608288 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:04.608603 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:04.608606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:04.608627 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:04.608780 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.608786 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.609559 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.609563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.610672 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.610685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.611544 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.611548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.611679 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.611703 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 08:35:04.611904 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 08:35:04.612008 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":76,"thread_start_us":103,"threads_started":1}
I20260807 08:35:04.612785 21298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "993bd5d07345421b9ea03dd7f31222a6" candidate_uuid: "39f50f5b25d34bd188b4c6dced28aceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e22386798041d695b3f4cbcd82f494" is_pre_election: true
I20260807 08:35:04.613250 21298 raft_consensus.cc:2454] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39f50f5b25d34bd188b4c6dced28aceb in term 0.
I20260807 08:35:04.613989 20799 leader_election.cc:302] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f50f5b25d34bd188b4c6dced28aceb, b7e22386798041d695b3f4cbcd82f494; no voters:
I20260807 08:35:04.613971 21099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "993bd5d07345421b9ea03dd7f31222a6" candidate_uuid: "39f50f5b25d34bd188b4c6dced28aceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ff8eef27064232b25342734abbc47c" is_pre_election: true
I20260807 08:35:04.614384 21099 raft_consensus.cc:2454] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39f50f5b25d34bd188b4c6dced28aceb in term 0.
I20260807 08:35:04.614398 21414 raft_consensus.cc:2781] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:35:04.614521 21414 raft_consensus.cc:482] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:35:04.614610 21414 raft_consensus.cc:3037] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:04.616168 21414 raft_consensus.cc:504] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } }
I20260807 08:35:04.616667 21414 leader_election.cc:288] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [CANDIDATE]: Term 1 election: Requested vote from peers b7e22386798041d695b3f4cbcd82f494 (127.9.174.3:36969), a8ff8eef27064232b25342734abbc47c (127.9.174.2:44939)
I20260807 08:35:04.617426 21099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "993bd5d07345421b9ea03dd7f31222a6" candidate_uuid: "39f50f5b25d34bd188b4c6dced28aceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ff8eef27064232b25342734abbc47c"
W20260807 08:35:04.617472 21155 tablet.cc:2369] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:35:04.617486 21298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "993bd5d07345421b9ea03dd7f31222a6" candidate_uuid: "39f50f5b25d34bd188b4c6dced28aceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e22386798041d695b3f4cbcd82f494"
I20260807 08:35:04.617626 21099 raft_consensus.cc:3037] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:04.617676 21298 raft_consensus.cc:3037] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:04.619508 21099 raft_consensus.cc:2454] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f50f5b25d34bd188b4c6dced28aceb in term 1.
I20260807 08:35:04.619508 21298 raft_consensus.cc:2454] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f50f5b25d34bd188b4c6dced28aceb in term 1.
I20260807 08:35:04.619983 20799 leader_election.cc:302] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f50f5b25d34bd188b4c6dced28aceb, b7e22386798041d695b3f4cbcd82f494; no voters:
I20260807 08:35:04.620391 21414 raft_consensus.cc:2781] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:35:04.620741 21414 raft_consensus.cc:686] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [term 1 LEADER]: Becoming Leader. State: Replica: 39f50f5b25d34bd188b4c6dced28aceb, State: Running, Role: LEADER
I20260807 08:35:04.621222 21414 consensus_queue.cc:237] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } }
I20260807 08:35:04.626003 20661 catalog_manager.cc:5495] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb reported cstate change: term changed from 0 to 1, leader changed from <none> to 39f50f5b25d34bd188b4c6dced28aceb (127.9.174.1). New cstate: current_term: 1 leader_uuid: "39f50f5b25d34bd188b4c6dced28aceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } health_report { overall_health: UNKNOWN } } }
I20260807 08:35:04.668852 21406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.664273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41770 (local address 127.9.174.1:33173)
0807 08:35:04.664472 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:04.664480 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:04.664499 (+ 19us) server_negotiation.cc:407] Connection header received
0807 08:35:04.664654 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:04.664657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:04.664733 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:04.664886 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:35:04.665233 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.665851 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.666897 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.667210 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.667340 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.667689 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:35:04.668349 (+ 660us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:35:04.668501 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:04.668553 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 08:35:04.668634 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 08:35:04.671947 21364 tablet.cc:2369] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:35:04.677506 21099 raft_consensus.cc:1261] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Refusing update from remote peer 39f50f5b25d34bd188b4c6dced28aceb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:35:04.677506 21298 raft_consensus.cc:1261] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494 [term 1 FOLLOWER]: Refusing update from remote peer 39f50f5b25d34bd188b4c6dced28aceb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:35:04.678434 21414 consensus_queue.cc:1035] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:35:04.678774 21420 consensus_queue.cc:1035] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:35:04.689689 21426 mvcc.cc:205] Tried to move back new op lower bound from 7315831622348881920 to 7315831622132047872. Current Snapshot: MvccSnapshot[applied={T|T < 7315831622348881920}]
I20260807 08:35:04.690395 21427 mvcc.cc:205] Tried to move back new op lower bound from 7315831622348881920 to 7315831622132047872. Current Snapshot: MvccSnapshot[applied={T|T < 7315831622348881920}]
I20260807 08:35:04.690454 21428 mvcc.cc:205] Tried to move back new op lower bound from 7315831622348881920 to 7315831622132047872. Current Snapshot: MvccSnapshot[applied={T|T < 7315831622348881920}]
I20260807 08:35:04.700410 20661 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:35:04.700726 20661 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:35:04.704317 20661 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d: Sending DeleteTablet for 3 replicas of tablet 993bd5d07345421b9ea03dd7f31222a6
I20260807 08:35:04.705659 21060 tablet_service.cc:1514] Processing DeleteTablet for tablet 993bd5d07345421b9ea03dd7f31222a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:04 UTC) from {username='oryx', principal='oryx/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:49690
I20260807 08:35:04.705680 21264 tablet_service.cc:1514] Processing DeleteTablet for tablet 993bd5d07345421b9ea03dd7f31222a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:04 UTC) from {username='oryx', principal='oryx/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:56768
I20260807 08:35:04.705796 20859 tablet_service.cc:1514] Processing DeleteTablet for tablet 993bd5d07345421b9ea03dd7f31222a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:04 UTC) from {username='oryx', principal='oryx/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:41762
I20260807 08:35:04.706508 21448 tablet_replica.cc:330] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb: stopping tablet replica
I20260807 08:35:04.706604 21443 tablet_replica.cc:330] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c: stopping tablet replica
I20260807 08:35:04.706606 20661 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:35:04.707150 20661 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:35:04.707165 21443 raft_consensus.cc:2229] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:35:04.707338 21448 raft_consensus.cc:2229] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:35:04.707624 21443 raft_consensus.cc:2258] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:04.708170 21448 raft_consensus.cc:2258] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:04.708187 21450 tablet_replica.cc:330] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494: stopping tablet replica
I20260807 08:35:04.708577 21450 raft_consensus.cc:2229] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:35:04.708945 21450 raft_consensus.cc:2258] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:04.710847 21448 ts_tablet_manager.cc:1905] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:35:04.711159 21450 ts_tablet_manager.cc:1905] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:35:04.714591 21448 ts_tablet_manager.cc:1918] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:35:04.714664 21450 ts_tablet_manager.cc:1918] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:35:04.714874 21450 log.cc:1196] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/993bd5d07345421b9ea03dd7f31222a6
I20260807 08:35:04.714833 21448 log.cc:1196] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/wal/wals/993bd5d07345421b9ea03dd7f31222a6
I20260807 08:35:04.715291 20859 tablet_service.cc:1467] Processing CreateTablet for tablet 1180f8d4324846c19e00e8c11aa3e937 (DEFAULT_TABLE table=test-table [id=50f27ef2d3c34da8ab7598ec30d0c5c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:04.715361 21450 ts_tablet_manager.cc:1939] T 993bd5d07345421b9ea03dd7f31222a6 P b7e22386798041d695b3f4cbcd82f494: Deleting consensus metadata
I20260807 08:35:04.715523 21448 ts_tablet_manager.cc:1939] T 993bd5d07345421b9ea03dd7f31222a6 P 39f50f5b25d34bd188b4c6dced28aceb: Deleting consensus metadata
I20260807 08:35:04.715655 20859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1180f8d4324846c19e00e8c11aa3e937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:04.717473 21264 tablet_service.cc:1467] Processing CreateTablet for tablet 1180f8d4324846c19e00e8c11aa3e937 (DEFAULT_TABLE table=test-table [id=50f27ef2d3c34da8ab7598ec30d0c5c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:04.717811 21264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1180f8d4324846c19e00e8c11aa3e937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:04.718199 20644 catalog_manager.cc:4841] TS b7e22386798041d695b3f4cbcd82f494 (127.9.174.3:36969): tablet 993bd5d07345421b9ea03dd7f31222a6 (table test-table [id=3361acb75dc44eddacbfa2af7fd835db]) successfully deleted
I20260807 08:35:04.718313 21060 tablet_service.cc:1467] Processing CreateTablet for tablet 1180f8d4324846c19e00e8c11aa3e937 (DEFAULT_TABLE table=test-table [id=50f27ef2d3c34da8ab7598ec30d0c5c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:04.718719 21060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1180f8d4324846c19e00e8c11aa3e937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:04.719422 21407 tablet_bootstrap.cc:492] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb: Bootstrap starting.
I20260807 08:35:04.720664 21407 tablet_bootstrap.cc:654] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:04.720754 21409 tablet_bootstrap.cc:492] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494: Bootstrap starting.
I20260807 08:35:04.721880 21409 tablet_bootstrap.cc:654] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:04.721940 21408 tablet_bootstrap.cc:492] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c: Bootstrap starting.
I20260807 08:35:04.722657 20642 catalog_manager.cc:4841] TS 39f50f5b25d34bd188b4c6dced28aceb (127.9.174.1:33173): tablet 993bd5d07345421b9ea03dd7f31222a6 (table test-table [id=3361acb75dc44eddacbfa2af7fd835db]) successfully deleted
I20260807 08:35:04.723248 21408 tablet_bootstrap.cc:654] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:04.723605 21407 tablet_bootstrap.cc:492] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb: No bootstrap required, opened a new log
I20260807 08:35:04.723724 21407 ts_tablet_manager.cc:1397] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260807 08:35:04.724350 21409 tablet_bootstrap.cc:492] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494: No bootstrap required, opened a new log
I20260807 08:35:04.724462 21409 ts_tablet_manager.cc:1397] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 08:35:04.724409 21407 raft_consensus.cc:348] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:04.724530 21407 raft_consensus.cc:374] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:04.724571 21407 raft_consensus.cc:729] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f50f5b25d34bd188b4c6dced28aceb, State: Initialized, Role: FOLLOWER
I20260807 08:35:04.724797 21407 consensus_queue.cc:260] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:04.724960 21443 ts_tablet_manager.cc:1905] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:35:04.725009 21409 raft_consensus.cc:348] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:04.725136 21409 raft_consensus.cc:374] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:04.725193 21409 raft_consensus.cc:729] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e22386798041d695b3f4cbcd82f494, State: Initialized, Role: FOLLOWER
I20260807 08:35:04.725338 21409 consensus_queue.cc:260] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:04.725651 21409 ts_tablet_manager.cc:1428] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 08:35:04.725703 21407 ts_tablet_manager.cc:1428] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:35:04.726881 21408 tablet_bootstrap.cc:492] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c: No bootstrap required, opened a new log
I20260807 08:35:04.727026 21408 ts_tablet_manager.cc:1397] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 08:35:04.727690 21408 raft_consensus.cc:348] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:04.727854 21408 raft_consensus.cc:374] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:04.727900 21408 raft_consensus.cc:729] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ff8eef27064232b25342734abbc47c, State: Initialized, Role: FOLLOWER
I20260807 08:35:04.728048 21408 consensus_queue.cc:260] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:04.729506 21408 ts_tablet_manager.cc:1428] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:35:04.729628 21443 ts_tablet_manager.cc:1918] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:35:04.729795 21443 log.cc:1196] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/993bd5d07345421b9ea03dd7f31222a6
I20260807 08:35:04.730307 21443 ts_tablet_manager.cc:1939] T 993bd5d07345421b9ea03dd7f31222a6 P a8ff8eef27064232b25342734abbc47c: Deleting consensus metadata
I20260807 08:35:04.732120 20644 catalog_manager.cc:4841] TS a8ff8eef27064232b25342734abbc47c (127.9.174.2:44939): tablet 993bd5d07345421b9ea03dd7f31222a6 (table test-table [id=3361acb75dc44eddacbfa2af7fd835db]) successfully deleted
I20260807 08:35:04.871480 21414 raft_consensus.cc:482] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:35:04.871717 21414 raft_consensus.cc:504] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:04.872411 21414 leader_election.cc:288] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ff8eef27064232b25342734abbc47c (127.9.174.2:44939), b7e22386798041d695b3f4cbcd82f494 (127.9.174.3:36969)
I20260807 08:35:04.873648 21099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1180f8d4324846c19e00e8c11aa3e937" candidate_uuid: "39f50f5b25d34bd188b4c6dced28aceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ff8eef27064232b25342734abbc47c" is_pre_election: true
I20260807 08:35:04.873904 21099 raft_consensus.cc:2454] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39f50f5b25d34bd188b4c6dced28aceb in term 0.
I20260807 08:35:04.874241 21298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1180f8d4324846c19e00e8c11aa3e937" candidate_uuid: "39f50f5b25d34bd188b4c6dced28aceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e22386798041d695b3f4cbcd82f494" is_pre_election: true
I20260807 08:35:04.874450 21298 raft_consensus.cc:2454] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39f50f5b25d34bd188b4c6dced28aceb in term 0.
I20260807 08:35:04.875126 20799 leader_election.cc:302] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f50f5b25d34bd188b4c6dced28aceb, a8ff8eef27064232b25342734abbc47c; no voters:
I20260807 08:35:04.875521 21414 raft_consensus.cc:2781] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:35:04.875602 21414 raft_consensus.cc:482] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:35:04.875658 21414 raft_consensus.cc:3037] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:04.877450 21414 raft_consensus.cc:504] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:04.877980 21414 leader_election.cc:288] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [CANDIDATE]: Term 1 election: Requested vote from peers a8ff8eef27064232b25342734abbc47c (127.9.174.2:44939), b7e22386798041d695b3f4cbcd82f494 (127.9.174.3:36969)
I20260807 08:35:04.878597 21099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1180f8d4324846c19e00e8c11aa3e937" candidate_uuid: "39f50f5b25d34bd188b4c6dced28aceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ff8eef27064232b25342734abbc47c"
I20260807 08:35:04.878791 21099 raft_consensus.cc:3037] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:04.878762 21298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1180f8d4324846c19e00e8c11aa3e937" candidate_uuid: "39f50f5b25d34bd188b4c6dced28aceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e22386798041d695b3f4cbcd82f494"
I20260807 08:35:04.878939 21298 raft_consensus.cc:3037] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:04.880553 21298 raft_consensus.cc:2454] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f50f5b25d34bd188b4c6dced28aceb in term 1.
I20260807 08:35:04.880652 21099 raft_consensus.cc:2454] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f50f5b25d34bd188b4c6dced28aceb in term 1.
I20260807 08:35:04.881182 20799 leader_election.cc:302] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f50f5b25d34bd188b4c6dced28aceb, a8ff8eef27064232b25342734abbc47c; no voters:
I20260807 08:35:04.881561 21414 raft_consensus.cc:2781] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:35:04.881785 21414 raft_consensus.cc:686] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [term 1 LEADER]: Becoming Leader. State: Replica: 39f50f5b25d34bd188b4c6dced28aceb, State: Running, Role: LEADER
I20260807 08:35:04.881995 21414 consensus_queue.cc:237] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:04.884668 20658 catalog_manager.cc:5495] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb reported cstate change: term changed from 0 to 1, leader changed from <none> to 39f50f5b25d34bd188b4c6dced28aceb (127.9.174.1). New cstate: current_term: 1 leader_uuid: "39f50f5b25d34bd188b4c6dced28aceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } health_report { overall_health: UNKNOWN } } }
Aug 07 08:35:04 dist-test-slave-h3ft krb5kdc[20606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091703, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.62@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM
I20260807 08:35:04.978474 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.966557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.9.174.62:39493)
0807 08:35:04.966770 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:04.966780 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:04.967161 (+ 381us) server_negotiation.cc:407] Connection header received
0807 08:35:04.967338 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:04.967341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:04.967389 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:04.967475 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:04.968644 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.969230 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.970737 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.971027 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.971166 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.974566 (+ 3400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:04.974570 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:04.974573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:04.974598 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:04.976307 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:04.976908 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:04.976915 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:04.976917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:04.977010 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:04.977339 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:04.977342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:04.977344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:04.977582 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:04.977715 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:04.978061 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 08:35:04.978234 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 08:35:04.978485 21462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:04.966511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:39493 (local address 127.0.0.1:38736)
0807 08:35:04.967027 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:04.967047 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:04.967232 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:04.967544 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:04.967548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:04.968297 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:04.968474 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.968487 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.969356 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.969362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.970493 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:04.970508 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:04.971189 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:04.971193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:04.971356 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:04.972440 (+ 1084us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:04.972455 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:04.974361 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:04.976463 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:04.976479 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:04.976488 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:04.976785 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:04.977126 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:04.977129 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:04.977131 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:04.977229 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:04.977715 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:04.977726 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:04.977866 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 08:35:04.978011 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 08:35:04.978317 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":109,"thread_start_us":186,"threads_started":1}
I20260807 08:35:04.984935 20658 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:38736:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 08:35:04.986006 20658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:35:04.992621 20859 tablet_service.cc:1467] Processing CreateTablet for tablet d403cedac27a4d74865c2c39fc4949f6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fecfc41ec1a54e1cbbff76eac1f14e93]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:35:04.992866 21264 tablet_service.cc:1467] Processing CreateTablet for tablet d403cedac27a4d74865c2c39fc4949f6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fecfc41ec1a54e1cbbff76eac1f14e93]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:35:04.993273 21264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d403cedac27a4d74865c2c39fc4949f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:04.993270 20859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d403cedac27a4d74865c2c39fc4949f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:04.993162 21060 tablet_service.cc:1467] Processing CreateTablet for tablet d403cedac27a4d74865c2c39fc4949f6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fecfc41ec1a54e1cbbff76eac1f14e93]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:35:04.993692 21060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d403cedac27a4d74865c2c39fc4949f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:04.997146 21409 tablet_bootstrap.cc:492] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494: Bootstrap starting.
I20260807 08:35:04.998368 21409 tablet_bootstrap.cc:654] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:04.999907 21409 tablet_bootstrap.cc:492] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494: No bootstrap required, opened a new log
I20260807 08:35:04.999890 21408 tablet_bootstrap.cc:492] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c: Bootstrap starting.
I20260807 08:35:05.000011 21409 ts_tablet_manager.cc:1397] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:35:05.000517 21409 raft_consensus.cc:348] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.000669 21409 raft_consensus.cc:374] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:05.000711 21409 raft_consensus.cc:729] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e22386798041d695b3f4cbcd82f494, State: Initialized, Role: FOLLOWER
I20260807 08:35:05.000877 21409 consensus_queue.cc:260] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.001199 21417 tablet_replica.cc:439] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } }
I20260807 08:35:05.001374 21417 tablet_replica.cc:442] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:35:05.001617 21408 tablet_bootstrap.cc:654] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:05.001673 21409 ts_tablet_manager.cc:1428] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:35:05.002023 21407 tablet_bootstrap.cc:492] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb: Bootstrap starting.
I20260807 08:35:05.003202 21407 tablet_bootstrap.cc:654] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:05.003623 21408 tablet_bootstrap.cc:492] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c: No bootstrap required, opened a new log
I20260807 08:35:05.003732 21408 ts_tablet_manager.cc:1397] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 08:35:05.004354 21408 raft_consensus.cc:348] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.004503 21408 raft_consensus.cc:374] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:05.004527 21408 raft_consensus.cc:729] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ff8eef27064232b25342734abbc47c, State: Initialized, Role: FOLLOWER
I20260807 08:35:05.004652 21408 consensus_queue.cc:260] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.004982 21408 ts_tablet_manager.cc:1428] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:35:05.004932 21415 tablet_replica.cc:439] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } }
I20260807 08:35:05.005101 21415 tablet_replica.cc:442] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:35:05.007398 21407 tablet_bootstrap.cc:492] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb: No bootstrap required, opened a new log
I20260807 08:35:05.007547 21407 ts_tablet_manager.cc:1397] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 08:35:05.008173 21407 raft_consensus.cc:348] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.008332 21407 raft_consensus.cc:374] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:05.008368 21407 raft_consensus.cc:729] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f50f5b25d34bd188b4c6dced28aceb, State: Initialized, Role: FOLLOWER
I20260807 08:35:05.008520 21407 consensus_queue.cc:260] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.008844 21414 tablet_replica.cc:439] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } }
I20260807 08:35:05.009002 21414 tablet_replica.cc:442] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:35:05.009276 21407 ts_tablet_manager.cc:1428] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:35:05.059312 21417 raft_consensus.cc:482] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:35:05.059576 21417 raft_consensus.cc:504] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.061152 21417 leader_election.cc:288] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39f50f5b25d34bd188b4c6dced28aceb (127.9.174.1:33173), a8ff8eef27064232b25342734abbc47c (127.9.174.2:44939)
I20260807 08:35:05.067049 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:05.061600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:33173 (local address 127.9.174.3:43851)
0807 08:35:05.062224 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:05.062245 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:05.062622 (+ 377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:05.063098 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:05.063101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:05.063119 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:05.063316 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:05.063325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.064222 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.064226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:05.065231 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:05.065244 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.066336 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.066341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:05.066509 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:05.066542 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 08:35:05.066639 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 08:35:05.066753 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":31,"thread_start_us":127,"threads_started":1}
I20260807 08:35:05.067085 21406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:05.062048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:43851 (local address 127.9.174.1:33173)
0807 08:35:05.062254 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:05.062261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:05.062420 (+ 159us) server_negotiation.cc:407] Connection header received
0807 08:35:05.062859 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:05.062864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:05.062956 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:05.063083 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:05.063461 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.064059 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.065363 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.066042 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.066251 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:05.066373 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:05.066685 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 08:35:05.066789 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 08:35:05.067225 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:05.062184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:60891 (local address 127.9.174.2:44939)
0807 08:35:05.062436 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:05.062442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:05.062543 (+ 101us) server_negotiation.cc:407] Connection header received
0807 08:35:05.062874 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:05.062878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:05.062939 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:05.063084 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:05.063469 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.064044 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.065719 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.066335 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.066459 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:05.066551 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:05.066904 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 08:35:05.067054 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260807 08:35:05.067095 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:05.061600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:44939 (local address 127.9.174.3:60891)
0807 08:35:05.062436 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:05.062456 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:05.062666 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:05.063081 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:05.063085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:05.063105 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:05.063324 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:05.063331 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.064182 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.064187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:05.065515 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:05.065527 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.066642 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.066646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:05.066758 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:05.066777 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 08:35:05.066851 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 08:35:05.066917 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":92,"spinlock_wait_cycles":15232,"thread_start_us":84,"threads_started":1}
I20260807 08:35:05.067997 21099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d403cedac27a4d74865c2c39fc4949f6" candidate_uuid: "b7e22386798041d695b3f4cbcd82f494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ff8eef27064232b25342734abbc47c" is_pre_election: true
I20260807 08:35:05.068120 20888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d403cedac27a4d74865c2c39fc4949f6" candidate_uuid: "b7e22386798041d695b3f4cbcd82f494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f50f5b25d34bd188b4c6dced28aceb" is_pre_election: true
I20260807 08:35:05.068252 21099 raft_consensus.cc:2454] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7e22386798041d695b3f4cbcd82f494 in term 0.
I20260807 08:35:05.068418 20888 raft_consensus.cc:2454] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7e22386798041d695b3f4cbcd82f494 in term 0.
I20260807 08:35:05.068825 21191 leader_election.cc:302] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ff8eef27064232b25342734abbc47c, b7e22386798041d695b3f4cbcd82f494; no voters:
I20260807 08:35:05.069236 21417 raft_consensus.cc:2781] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:35:05.069307 21417 raft_consensus.cc:482] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:35:05.069337 21417 raft_consensus.cc:3037] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:05.070706 21417 raft_consensus.cc:504] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.071336 21417 leader_election.cc:288] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [CANDIDATE]: Term 1 election: Requested vote from peers 39f50f5b25d34bd188b4c6dced28aceb (127.9.174.1:33173), a8ff8eef27064232b25342734abbc47c (127.9.174.2:44939)
I20260807 08:35:05.071854 20888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d403cedac27a4d74865c2c39fc4949f6" candidate_uuid: "b7e22386798041d695b3f4cbcd82f494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f50f5b25d34bd188b4c6dced28aceb"
I20260807 08:35:05.071877 21099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d403cedac27a4d74865c2c39fc4949f6" candidate_uuid: "b7e22386798041d695b3f4cbcd82f494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ff8eef27064232b25342734abbc47c"
I20260807 08:35:05.072067 20888 raft_consensus.cc:3037] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:05.072074 21099 raft_consensus.cc:3037] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:05.073601 20888 raft_consensus.cc:2454] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7e22386798041d695b3f4cbcd82f494 in term 1.
I20260807 08:35:05.073613 21099 raft_consensus.cc:2454] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7e22386798041d695b3f4cbcd82f494 in term 1.
I20260807 08:35:05.074005 21188 leader_election.cc:302] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f50f5b25d34bd188b4c6dced28aceb, b7e22386798041d695b3f4cbcd82f494; no voters:
I20260807 08:35:05.074244 21417 raft_consensus.cc:2781] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:35:05.074503 21417 raft_consensus.cc:686] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [term 1 LEADER]: Becoming Leader. State: Replica: b7e22386798041d695b3f4cbcd82f494, State: Running, Role: LEADER
I20260807 08:35:05.074864 21417 consensus_queue.cc:237] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.076659 21469 tablet_replica.cc:439] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494: TxnStatusTablet state changed. Reason: New leader b7e22386798041d695b3f4cbcd82f494. Latest consensus state: current_term: 1 leader_uuid: "b7e22386798041d695b3f4cbcd82f494" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } }
I20260807 08:35:05.076815 21469 tablet_replica.cc:442] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:35:05.077157 21472 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 08:35:05.077343 21472 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 08:35:05.078336 20661 catalog_manager.cc:5495] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7e22386798041d695b3f4cbcd82f494 (127.9.174.3). New cstate: current_term: 1 leader_uuid: "b7e22386798041d695b3f4cbcd82f494" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } health_report { overall_health: HEALTHY } } }
I20260807 08:35:05.106148 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:05.100629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56772 (local address 127.9.174.3:36969)
0807 08:35:05.100975 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:05.100980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:05.101005 (+ 25us) server_negotiation.cc:407] Connection header received
0807 08:35:05.101112 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:05.101115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:05.101192 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:05.101303 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:35:05.101708 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.102365 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.103832 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.104120 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.104237 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:05.104707 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:35:05.105468 (+ 761us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:35:05.105587 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:05.105888 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 08:35:05.105993 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260807 08:35:05.106113 21462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:05.100446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:36969 (local address 127.0.0.1:56772)
0807 08:35:05.100719 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:05.100741 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:05.100945 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:05.101356 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:05.101359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:05.101382 (+ 23us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:35:05.101535 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:05.101542 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.102515 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.102519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:05.103615 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:05.103632 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.104285 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.104289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:05.104394 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:05.104488 (+ 94us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:35:05.105599 (+ 1111us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:35:05.105620 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 08:35:05.105713 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 08:35:05.105874 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 08:35:05.110968 21099 raft_consensus.cc:1261] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Refusing update from remote peer b7e22386798041d695b3f4cbcd82f494: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:35:05.111279 20888 raft_consensus.cc:1261] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb [term 1 FOLLOWER]: Refusing update from remote peer b7e22386798041d695b3f4cbcd82f494: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:35:05.111800 21469 consensus_queue.cc:1035] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:35:05.112069 21417 consensus_queue.cc:1035] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:35:05.115293 21414 tablet_replica.cc:439] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb: TxnStatusTablet state changed. Reason: New leader b7e22386798041d695b3f4cbcd82f494. Latest consensus state: current_term: 1 leader_uuid: "b7e22386798041d695b3f4cbcd82f494" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } }
I20260807 08:35:05.115507 21414 tablet_replica.cc:442] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:35:05.118196 21417 tablet_replica.cc:439] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7e22386798041d695b3f4cbcd82f494" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } }
I20260807 08:35:05.118389 21417 tablet_replica.cc:442] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:35:05.118448 21415 tablet_replica.cc:439] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c: TxnStatusTablet state changed. Reason: New leader b7e22386798041d695b3f4cbcd82f494. Latest consensus state: current_term: 1 leader_uuid: "b7e22386798041d695b3f4cbcd82f494" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } }
I20260807 08:35:05.118639 21415 tablet_replica.cc:442] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:35:05.120497 21417 tablet_replica.cc:439] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7e22386798041d695b3f4cbcd82f494" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } }
I20260807 08:35:05.120694 21417 tablet_replica.cc:442] T d403cedac27a4d74865c2c39fc4949f6 P b7e22386798041d695b3f4cbcd82f494: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:35:05.122166 21414 tablet_replica.cc:439] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7e22386798041d695b3f4cbcd82f494" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } }
I20260807 08:35:05.122373 21414 tablet_replica.cc:442] T d403cedac27a4d74865c2c39fc4949f6 P 39f50f5b25d34bd188b4c6dced28aceb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:35:05.122490 21415 tablet_replica.cc:439] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7e22386798041d695b3f4cbcd82f494" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } } }
I20260807 08:35:05.122648 21415 tablet_replica.cc:442] T d403cedac27a4d74865c2c39fc4949f6 P a8ff8eef27064232b25342734abbc47c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:35:05.147298 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:05.140147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:46135 (local address 127.9.174.3:36969)
0807 08:35:05.140322 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:05.140327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:05.143561 (+ 3234us) server_negotiation.cc:407] Connection header received
0807 08:35:05.143788 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:05.143792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:05.143891 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:05.143992 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:35:05.144310 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.144947 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.145936 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.146177 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.146301 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:05.146675 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:35:05.146776 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:35:05.146872 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:05.147033 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 08:35:05.147102 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 08:35:05.147300 21480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:05.140033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:36969 (local address 127.9.174.1:46135)
0807 08:35:05.143411 (+ 3378us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:05.143431 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:05.143639 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:05.144012 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:05.144015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:05.144025 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:35:05.144171 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:05.144177 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.145084 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.145088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:05.145789 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:05.145798 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.146320 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.146324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:05.146424 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:05.146494 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:35:05.146874 (+ 380us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:35:05.146885 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 08:35:05.146974 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 08:35:05.147113 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3188,"thread_start_us":169,"threads_started":1}
I20260807 08:35:05.158476 21099 raft_consensus.cc:1261] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Refusing update from remote peer 39f50f5b25d34bd188b4c6dced28aceb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:35:05.158478 21298 raft_consensus.cc:1261] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494 [term 1 FOLLOWER]: Refusing update from remote peer 39f50f5b25d34bd188b4c6dced28aceb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:35:05.159078 21414 consensus_queue.cc:1035] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:35:05.159309 21434 consensus_queue.cc:1035] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:35:05.191823 21406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:05.187353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:36141 (local address 127.9.174.1:33173)
0807 08:35:05.187613 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:05.187619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:05.187775 (+ 156us) server_negotiation.cc:407] Connection header received
0807 08:35:05.187981 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:05.187986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:05.188058 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:05.188152 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:35:05.188484 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.189348 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.190277 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.190629 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.190740 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:05.191226 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:35:05.191335 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:35:05.191408 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:05.191559 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 08:35:05.191633 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260807 08:35:05.192298 21489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:05.187254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:33173 (local address 127.9.174.3:36141)
0807 08:35:05.187670 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:05.187692 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:05.187848 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:05.188163 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:05.188166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:05.188175 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:35:05.188341 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:05.188349 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.189455 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.189458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:05.190144 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:05.190154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.190780 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.190784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:05.190868 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:05.190929 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:35:05.191453 (+ 524us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:35:05.191461 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 08:35:05.192033 (+ 572us) client_negotiation.cc:240] Negotiation successful
0807 08:35:05.192139 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":157,"threads_started":1}
I20260807 08:35:05.222393 20661 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:35:05.222693 20661 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:35:05.226022 20661 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d: Sending DeleteTablet for 3 replicas of tablet 1180f8d4324846c19e00e8c11aa3e937
I20260807 08:35:05.226990 21060 tablet_service.cc:1514] Processing DeleteTablet for tablet 1180f8d4324846c19e00e8c11aa3e937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:05 UTC) from {username='oryx', principal='oryx/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:49690
I20260807 08:35:05.227135 20859 tablet_service.cc:1514] Processing DeleteTablet for tablet 1180f8d4324846c19e00e8c11aa3e937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:05 UTC) from {username='oryx', principal='oryx/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:41762
I20260807 08:35:05.227298 21443 tablet_replica.cc:330] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c: stopping tablet replica
I20260807 08:35:05.227485 21443 raft_consensus.cc:2229] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:35:05.227625 21494 tablet_replica.cc:330] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb: stopping tablet replica
I20260807 08:35:05.227612 21264 tablet_service.cc:1514] Processing DeleteTablet for tablet 1180f8d4324846c19e00e8c11aa3e937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:05 UTC) from {username='oryx', principal='oryx/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:56768
I20260807 08:35:05.227686 21443 raft_consensus.cc:2258] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:05.227908 21494 raft_consensus.cc:2229] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:35:05.228240 21494 raft_consensus.cc:2258] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:05.228675 21495 tablet_replica.cc:330] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494: stopping tablet replica
I20260807 08:35:05.228792 21443 ts_tablet_manager.cc:1905] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:35:05.228842 21495 raft_consensus.cc:2229] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:35:05.229089 21495 raft_consensus.cc:2258] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:05.230034 21495 ts_tablet_manager.cc:1905] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:35:05.234166 21443 ts_tablet_manager.cc:1918] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:35:05.234308 21443 log.cc:1196] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/1180f8d4324846c19e00e8c11aa3e937
I20260807 08:35:05.234648 21443 ts_tablet_manager.cc:1939] T 1180f8d4324846c19e00e8c11aa3e937 P a8ff8eef27064232b25342734abbc47c: Deleting consensus metadata
I20260807 08:35:05.234812 21495 ts_tablet_manager.cc:1918] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:35:05.234910 21495 log.cc:1196] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/1180f8d4324846c19e00e8c11aa3e937
I20260807 08:35:05.235273 21495 ts_tablet_manager.cc:1939] T 1180f8d4324846c19e00e8c11aa3e937 P b7e22386798041d695b3f4cbcd82f494: Deleting consensus metadata
I20260807 08:35:05.236126 20644 catalog_manager.cc:4841] TS a8ff8eef27064232b25342734abbc47c (127.9.174.2:44939): tablet 1180f8d4324846c19e00e8c11aa3e937 (table test-table [id=50f27ef2d3c34da8ab7598ec30d0c5c5]) successfully deleted
I20260807 08:35:05.236646 20644 catalog_manager.cc:4841] TS b7e22386798041d695b3f4cbcd82f494 (127.9.174.3:36969): tablet 1180f8d4324846c19e00e8c11aa3e937 (table test-table [id=50f27ef2d3c34da8ab7598ec30d0c5c5]) successfully deleted
I20260807 08:35:05.236827 21494 ts_tablet_manager.cc:1905] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:35:05.241356 21494 ts_tablet_manager.cc:1918] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:35:05.241503 21494 log.cc:1196] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/wal/wals/1180f8d4324846c19e00e8c11aa3e937
I20260807 08:35:05.241870 21494 ts_tablet_manager.cc:1939] T 1180f8d4324846c19e00e8c11aa3e937 P 39f50f5b25d34bd188b4c6dced28aceb: Deleting consensus metadata
I20260807 08:35:05.243166 20642 catalog_manager.cc:4841] TS 39f50f5b25d34bd188b4c6dced28aceb (127.9.174.1:33173): tablet 1180f8d4324846c19e00e8c11aa3e937 (table test-table [id=50f27ef2d3c34da8ab7598ec30d0c5c5]) successfully deleted
Aug 07 08:35:05 dist-test-slave-h3ft krb5kdc[20606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:35:05 dist-test-slave-h3ft krb5kdc[20606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:05 dist-test-slave-h3ft krb5kdc[20606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 08:35:05.270251 20735 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:05.261784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38744 (local address 127.9.174.62:39493)
0807 08:35:05.261969 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:05.261975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:05.262276 (+ 301us) server_negotiation.cc:407] Connection header received
0807 08:35:05.262500 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:05.262507 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:05.262574 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:05.262680 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:05.263720 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.264330 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.265424 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.265693 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.265826 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:05.270015 (+ 4189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38744: BlockingRecv error: recv got EOF from 127.0.0.1:38744 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 08:35:05 dist-test-slave-h3ft krb5kdc[20606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM
I20260807 08:35:05.283210 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:05.272679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38748 (local address 127.9.174.62:39493)
0807 08:35:05.272843 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:05.272847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:05.272860 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:35:05.272913 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:05.272915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:05.272961 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:05.273064 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:05.273824 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.274406 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.275531 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.275782 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.275904 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:05.278929 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:05.278933 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:05.278936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:05.278974 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:05.280990 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:05.281599 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:05.281603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:05.281606 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:05.281710 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:05.282017 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:05.282020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:05.282022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:05.282334 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:05.282502 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:05.282773 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 08:35:05.282931 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 08:35:05.286693 20661 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:35:05.287410 20661 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:35:05.293514 20859 tablet_service.cc:1467] Processing CreateTablet for tablet 8e15e88569e5453885cf9dfa4b3021da (DEFAULT_TABLE table=test-table [id=c76e5f226cee4d70935b29bae798c37c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:05.293669 21264 tablet_service.cc:1467] Processing CreateTablet for tablet 8e15e88569e5453885cf9dfa4b3021da (DEFAULT_TABLE table=test-table [id=c76e5f226cee4d70935b29bae798c37c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:05.293959 20859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e15e88569e5453885cf9dfa4b3021da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:05.294072 21264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e15e88569e5453885cf9dfa4b3021da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:05.297269 21407 tablet_bootstrap.cc:492] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb: Bootstrap starting.
I20260807 08:35:05.297314 21409 tablet_bootstrap.cc:492] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494: Bootstrap starting.
I20260807 08:35:05.297740 21060 tablet_service.cc:1467] Processing CreateTablet for tablet 8e15e88569e5453885cf9dfa4b3021da (DEFAULT_TABLE table=test-table [id=c76e5f226cee4d70935b29bae798c37c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:05.298117 21060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e15e88569e5453885cf9dfa4b3021da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:05.298293 21409 tablet_bootstrap.cc:654] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:05.298416 21407 tablet_bootstrap.cc:654] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:05.299647 21409 tablet_bootstrap.cc:492] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494: No bootstrap required, opened a new log
I20260807 08:35:05.299764 21409 ts_tablet_manager.cc:1397] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:35:05.300300 21409 raft_consensus.cc:348] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.300470 21409 raft_consensus.cc:374] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:05.300508 21409 raft_consensus.cc:729] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e22386798041d695b3f4cbcd82f494, State: Initialized, Role: FOLLOWER
I20260807 08:35:05.300664 21409 consensus_queue.cc:260] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.300752 21407 tablet_bootstrap.cc:492] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb: No bootstrap required, opened a new log
I20260807 08:35:05.300846 21407 ts_tablet_manager.cc:1397] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 08:35:05.301007 21409 ts_tablet_manager.cc:1428] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:35:05.301451 21408 tablet_bootstrap.cc:492] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c: Bootstrap starting.
I20260807 08:35:05.301349 21407 raft_consensus.cc:348] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.301496 21407 raft_consensus.cc:374] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:05.301529 21407 raft_consensus.cc:729] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f50f5b25d34bd188b4c6dced28aceb, State: Initialized, Role: FOLLOWER
I20260807 08:35:05.301662 21407 consensus_queue.cc:260] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.301957 21407 ts_tablet_manager.cc:1428] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:35:05.302508 21408 tablet_bootstrap.cc:654] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:05.304170 21408 tablet_bootstrap.cc:492] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c: No bootstrap required, opened a new log
I20260807 08:35:05.304315 21408 ts_tablet_manager.cc:1397] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:35:05.305459 21408 raft_consensus.cc:348] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.305591 21408 raft_consensus.cc:374] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:05.305615 21408 raft_consensus.cc:729] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ff8eef27064232b25342734abbc47c, State: Initialized, Role: FOLLOWER
I20260807 08:35:05.305765 21408 consensus_queue.cc:260] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.306092 21408 ts_tablet_manager.cc:1428] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 08:35:05.584558 21486 raft_consensus.cc:482] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:35:05.584805 21486 raft_consensus.cc:504] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.585562 21486 leader_election.cc:288] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39f50f5b25d34bd188b4c6dced28aceb (127.9.174.1:33173), a8ff8eef27064232b25342734abbc47c (127.9.174.2:44939)
I20260807 08:35:05.586292 21099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e15e88569e5453885cf9dfa4b3021da" candidate_uuid: "b7e22386798041d695b3f4cbcd82f494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ff8eef27064232b25342734abbc47c" is_pre_election: true
I20260807 08:35:05.586302 20888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e15e88569e5453885cf9dfa4b3021da" candidate_uuid: "b7e22386798041d695b3f4cbcd82f494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f50f5b25d34bd188b4c6dced28aceb" is_pre_election: true
I20260807 08:35:05.586534 21099 raft_consensus.cc:2454] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7e22386798041d695b3f4cbcd82f494 in term 0.
I20260807 08:35:05.586544 20888 raft_consensus.cc:2454] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7e22386798041d695b3f4cbcd82f494 in term 0.
I20260807 08:35:05.587033 21191 leader_election.cc:302] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ff8eef27064232b25342734abbc47c, b7e22386798041d695b3f4cbcd82f494; no voters:
I20260807 08:35:05.587340 21486 raft_consensus.cc:2781] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:35:05.587417 21486 raft_consensus.cc:482] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:35:05.587491 21486 raft_consensus.cc:3037] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:05.588956 21486 raft_consensus.cc:504] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.589535 21486 leader_election.cc:288] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [CANDIDATE]: Term 1 election: Requested vote from peers 39f50f5b25d34bd188b4c6dced28aceb (127.9.174.1:33173), a8ff8eef27064232b25342734abbc47c (127.9.174.2:44939)
I20260807 08:35:05.590304 21099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e15e88569e5453885cf9dfa4b3021da" candidate_uuid: "b7e22386798041d695b3f4cbcd82f494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ff8eef27064232b25342734abbc47c"
I20260807 08:35:05.590305 20888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e15e88569e5453885cf9dfa4b3021da" candidate_uuid: "b7e22386798041d695b3f4cbcd82f494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f50f5b25d34bd188b4c6dced28aceb"
I20260807 08:35:05.590516 21099 raft_consensus.cc:3037] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:05.590523 20888 raft_consensus.cc:3037] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:05.592489 20888 raft_consensus.cc:2454] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7e22386798041d695b3f4cbcd82f494 in term 1.
I20260807 08:35:05.592612 21099 raft_consensus.cc:2454] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7e22386798041d695b3f4cbcd82f494 in term 1.
I20260807 08:35:05.593417 21188 leader_election.cc:302] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f50f5b25d34bd188b4c6dced28aceb, b7e22386798041d695b3f4cbcd82f494; no voters:
I20260807 08:35:05.593971 21486 raft_consensus.cc:2781] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:35:05.594108 21486 raft_consensus.cc:686] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [term 1 LEADER]: Becoming Leader. State: Replica: b7e22386798041d695b3f4cbcd82f494, State: Running, Role: LEADER
I20260807 08:35:05.594318 21486 consensus_queue.cc:237] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.601621 20659 catalog_manager.cc:5495] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7e22386798041d695b3f4cbcd82f494 (127.9.174.3). New cstate: current_term: 1 leader_uuid: "b7e22386798041d695b3f4cbcd82f494" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } health_report { overall_health: HEALTHY } } }
I20260807 08:35:05.684635 21521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:05.680086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56782 (local address 127.9.174.3:36969)
0807 08:35:05.680407 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:05.680411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:05.680425 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:35:05.680488 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:05.680491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:05.680584 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:05.680727 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:35:05.681082 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.681889 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.682928 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.683399 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.683565 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:05.683970 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:35:05.684116 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:35:05.684220 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:05.684323 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 08:35:05.684403 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":90,"threads_started":1}
I20260807 08:35:05.688886 21099 raft_consensus.cc:1261] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Refusing update from remote peer b7e22386798041d695b3f4cbcd82f494: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:35:05.689162 20888 raft_consensus.cc:1261] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb [term 1 FOLLOWER]: Refusing update from remote peer b7e22386798041d695b3f4cbcd82f494: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:35:05.689785 21486 consensus_queue.cc:1035] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:35:05.689989 21417 consensus_queue.cc:1035] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:35:05.704623 20661 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:35:05.704893 20661 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:35:05.708039 20661 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d: Sending DeleteTablet for 3 replicas of tablet 8e15e88569e5453885cf9dfa4b3021da
I20260807 08:35:05.709095 21060 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e15e88569e5453885cf9dfa4b3021da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:05 UTC) from {username='oryx', principal='oryx/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:49690
I20260807 08:35:05.709100 20859 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e15e88569e5453885cf9dfa4b3021da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:05 UTC) from {username='oryx', principal='oryx/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:41762
I20260807 08:35:05.709394 21494 tablet_replica.cc:330] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb: stopping tablet replica
I20260807 08:35:05.709393 21443 tablet_replica.cc:330] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c: stopping tablet replica
I20260807 08:35:05.709571 21494 raft_consensus.cc:2229] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:35:05.709585 21443 raft_consensus.cc:2229] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:35:05.709782 21494 raft_consensus.cc:2258] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:05.709785 21443 raft_consensus.cc:2258] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:05.710706 21264 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e15e88569e5453885cf9dfa4b3021da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:05 UTC) from {username='oryx', principal='oryx/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:56768
I20260807 08:35:05.710918 21443 ts_tablet_manager.cc:1905] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:35:05.711045 21495 tablet_replica.cc:330] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494: stopping tablet replica
I20260807 08:35:05.711203 21495 raft_consensus.cc:2229] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:35:05.711280 21494 ts_tablet_manager.cc:1905] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:35:05.711738 21495 raft_consensus.cc:2258] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:05.712853 21495 ts_tablet_manager.cc:1905] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:35:05.714645 21443 ts_tablet_manager.cc:1918] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:35:05.714666 21494 ts_tablet_manager.cc:1918] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:35:05.714757 21443 log.cc:1196] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/8e15e88569e5453885cf9dfa4b3021da
I20260807 08:35:05.714762 21494 log.cc:1196] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-0/wal/wals/8e15e88569e5453885cf9dfa4b3021da
I20260807 08:35:05.715122 21494 ts_tablet_manager.cc:1939] T 8e15e88569e5453885cf9dfa4b3021da P 39f50f5b25d34bd188b4c6dced28aceb: Deleting consensus metadata
I20260807 08:35:05.715127 21443 ts_tablet_manager.cc:1939] T 8e15e88569e5453885cf9dfa4b3021da P a8ff8eef27064232b25342734abbc47c: Deleting consensus metadata
I20260807 08:35:05.716259 20642 catalog_manager.cc:4841] TS 39f50f5b25d34bd188b4c6dced28aceb (127.9.174.1:33173): tablet 8e15e88569e5453885cf9dfa4b3021da (table test-table [id=c76e5f226cee4d70935b29bae798c37c]) successfully deleted
I20260807 08:35:05.716259 20644 catalog_manager.cc:4841] TS a8ff8eef27064232b25342734abbc47c (127.9.174.2:44939): tablet 8e15e88569e5453885cf9dfa4b3021da (table test-table [id=c76e5f226cee4d70935b29bae798c37c]) successfully deleted
I20260807 08:35:05.717344 21495 ts_tablet_manager.cc:1918] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:35:05.717453 21495 log.cc:1196] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/8e15e88569e5453885cf9dfa4b3021da
I20260807 08:35:05.717798 21495 ts_tablet_manager.cc:1939] T 8e15e88569e5453885cf9dfa4b3021da P b7e22386798041d695b3f4cbcd82f494: Deleting consensus metadata
I20260807 08:35:05.717962 20661 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:35:05.718626 20661 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:35:05.718988 20644 catalog_manager.cc:4841] TS b7e22386798041d695b3f4cbcd82f494 (127.9.174.3:36969): tablet 8e15e88569e5453885cf9dfa4b3021da (table test-table [id=c76e5f226cee4d70935b29bae798c37c]) successfully deleted
I20260807 08:35:05.726902 20859 tablet_service.cc:1467] Processing CreateTablet for tablet fd0a6391bc474ee5aaf39c6589054b5f (DEFAULT_TABLE table=test-table [id=a01bf592bd7a4a6583e64386b1e388fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:05.726902 21060 tablet_service.cc:1467] Processing CreateTablet for tablet fd0a6391bc474ee5aaf39c6589054b5f (DEFAULT_TABLE table=test-table [id=a01bf592bd7a4a6583e64386b1e388fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:05.727547 20859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd0a6391bc474ee5aaf39c6589054b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:05.727542 21060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd0a6391bc474ee5aaf39c6589054b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:05.731137 21407 tablet_bootstrap.cc:492] T fd0a6391bc474ee5aaf39c6589054b5f P 39f50f5b25d34bd188b4c6dced28aceb: Bootstrap starting.
I20260807 08:35:05.731989 21264 tablet_service.cc:1467] Processing CreateTablet for tablet fd0a6391bc474ee5aaf39c6589054b5f (DEFAULT_TABLE table=test-table [id=a01bf592bd7a4a6583e64386b1e388fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:05.732378 21264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd0a6391bc474ee5aaf39c6589054b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:05.732759 21407 tablet_bootstrap.cc:654] T fd0a6391bc474ee5aaf39c6589054b5f P 39f50f5b25d34bd188b4c6dced28aceb: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:05.733671 21408 tablet_bootstrap.cc:492] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c: Bootstrap starting.
I20260807 08:35:05.734936 21408 tablet_bootstrap.cc:654] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:05.736272 21407 tablet_bootstrap.cc:492] T fd0a6391bc474ee5aaf39c6589054b5f P 39f50f5b25d34bd188b4c6dced28aceb: No bootstrap required, opened a new log
I20260807 08:35:05.736414 21407 ts_tablet_manager.cc:1397] T fd0a6391bc474ee5aaf39c6589054b5f P 39f50f5b25d34bd188b4c6dced28aceb: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260807 08:35:05.736732 21409 tablet_bootstrap.cc:492] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494: Bootstrap starting.
I20260807 08:35:05.736994 21407 raft_consensus.cc:348] T fd0a6391bc474ee5aaf39c6589054b5f P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.737144 21407 raft_consensus.cc:374] T fd0a6391bc474ee5aaf39c6589054b5f P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:05.737176 21407 raft_consensus.cc:729] T fd0a6391bc474ee5aaf39c6589054b5f P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f50f5b25d34bd188b4c6dced28aceb, State: Initialized, Role: FOLLOWER
I20260807 08:35:05.737376 21407 consensus_queue.cc:260] T fd0a6391bc474ee5aaf39c6589054b5f P 39f50f5b25d34bd188b4c6dced28aceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.737838 21407 ts_tablet_manager.cc:1428] T fd0a6391bc474ee5aaf39c6589054b5f P 39f50f5b25d34bd188b4c6dced28aceb: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 08:35:05.737977 21409 tablet_bootstrap.cc:654] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:05.738968 21408 tablet_bootstrap.cc:492] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c: No bootstrap required, opened a new log
I20260807 08:35:05.739147 21408 ts_tablet_manager.cc:1397] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260807 08:35:05.739744 21408 raft_consensus.cc:348] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.739890 21408 raft_consensus.cc:374] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:05.739883 21409 tablet_bootstrap.cc:492] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494: No bootstrap required, opened a new log
I20260807 08:35:05.739924 21408 raft_consensus.cc:729] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ff8eef27064232b25342734abbc47c, State: Initialized, Role: FOLLOWER
I20260807 08:35:05.739985 21409 ts_tablet_manager.cc:1397] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:35:05.740085 21408 consensus_queue.cc:260] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.740382 21408 ts_tablet_manager.cc:1428] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:35:05.740492 21409 raft_consensus.cc:348] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.740615 21409 raft_consensus.cc:374] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:05.740644 21409 raft_consensus.cc:729] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e22386798041d695b3f4cbcd82f494, State: Initialized, Role: FOLLOWER
I20260807 08:35:05.742178 21409 consensus_queue.cc:260] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.742391 21486 raft_consensus.cc:482] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:35:05.742483 21486 raft_consensus.cc:504] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.742971 21486 leader_election.cc:288] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39f50f5b25d34bd188b4c6dced28aceb (127.9.174.1:33173), a8ff8eef27064232b25342734abbc47c (127.9.174.2:44939)
I20260807 08:35:05.743587 21409 ts_tablet_manager.cc:1428] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260807 08:35:05.743471 21099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0a6391bc474ee5aaf39c6589054b5f" candidate_uuid: "b7e22386798041d695b3f4cbcd82f494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ff8eef27064232b25342734abbc47c" is_pre_election: true
I20260807 08:35:05.743794 21099 raft_consensus.cc:2454] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7e22386798041d695b3f4cbcd82f494 in term 0.
I20260807 08:35:05.744248 20888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0a6391bc474ee5aaf39c6589054b5f" candidate_uuid: "b7e22386798041d695b3f4cbcd82f494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f50f5b25d34bd188b4c6dced28aceb" is_pre_election: true
I20260807 08:35:05.744351 21191 leader_election.cc:302] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ff8eef27064232b25342734abbc47c, b7e22386798041d695b3f4cbcd82f494; no voters:
I20260807 08:35:05.744477 20888 raft_consensus.cc:2454] T fd0a6391bc474ee5aaf39c6589054b5f P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7e22386798041d695b3f4cbcd82f494 in term 0.
I20260807 08:35:05.744632 21417 raft_consensus.cc:2781] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:35:05.744733 21417 raft_consensus.cc:482] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:35:05.744788 21417 raft_consensus.cc:3037] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:05.746891 21417 raft_consensus.cc:504] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.747489 21417 leader_election.cc:288] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [CANDIDATE]: Term 1 election: Requested vote from peers 39f50f5b25d34bd188b4c6dced28aceb (127.9.174.1:33173), a8ff8eef27064232b25342734abbc47c (127.9.174.2:44939)
I20260807 08:35:05.748051 20888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0a6391bc474ee5aaf39c6589054b5f" candidate_uuid: "b7e22386798041d695b3f4cbcd82f494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f50f5b25d34bd188b4c6dced28aceb"
I20260807 08:35:05.748256 20888 raft_consensus.cc:3037] T fd0a6391bc474ee5aaf39c6589054b5f P 39f50f5b25d34bd188b4c6dced28aceb [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:05.748772 21099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0a6391bc474ee5aaf39c6589054b5f" candidate_uuid: "b7e22386798041d695b3f4cbcd82f494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ff8eef27064232b25342734abbc47c"
I20260807 08:35:05.748948 21099 raft_consensus.cc:3037] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:05.750435 20888 raft_consensus.cc:2454] T fd0a6391bc474ee5aaf39c6589054b5f P 39f50f5b25d34bd188b4c6dced28aceb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7e22386798041d695b3f4cbcd82f494 in term 1.
I20260807 08:35:05.750622 21099 raft_consensus.cc:2454] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7e22386798041d695b3f4cbcd82f494 in term 1.
I20260807 08:35:05.751057 21191 leader_election.cc:302] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 39f50f5b25d34bd188b4c6dced28aceb, a8ff8eef27064232b25342734abbc47c, b7e22386798041d695b3f4cbcd82f494; no voters:
I20260807 08:35:05.751333 21417 raft_consensus.cc:2781] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:35:05.751448 21417 raft_consensus.cc:686] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [term 1 LEADER]: Becoming Leader. State: Replica: b7e22386798041d695b3f4cbcd82f494, State: Running, Role: LEADER
I20260807 08:35:05.751627 21417 consensus_queue.cc:237] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } }
I20260807 08:35:05.753875 20661 catalog_manager.cc:5495] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7e22386798041d695b3f4cbcd82f494 (127.9.174.3). New cstate: current_term: 1 leader_uuid: "b7e22386798041d695b3f4cbcd82f494" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7e22386798041d695b3f4cbcd82f494" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 36969 } health_report { overall_health: HEALTHY } } }
I20260807 08:35:05.787333 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:05.782631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:36969 (local address 127.9.174.3:56139)
0807 08:35:05.783060 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:05.783092 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:05.783231 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:05.783575 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:05.783578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:05.783586 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:35:05.783811 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:05.783820 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.785034 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.785037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:05.785724 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:05.785734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.786464 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.786468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:05.786553 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:05.786590 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:35:05.786918 (+ 328us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:35:05.786926 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 08:35:05.787014 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 08:35:05.787121 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":43,"thread_start_us":163,"threads_started":1}
I20260807 08:35:05.787333 21521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:05.782699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:56139 (local address 127.9.174.3:36969)
0807 08:35:05.782878 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:05.782884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:05.783190 (+ 306us) server_negotiation.cc:407] Connection header received
0807 08:35:05.783353 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:05.783355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:05.783447 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:05.783565 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:35:05.784001 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.784871 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.786054 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:05.786314 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:05.786441 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:05.786718 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:35:05.786823 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:35:05.786916 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:05.787022 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 08:35:05.787103 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":63}
I20260807 08:35:05.798460 20888 raft_consensus.cc:1261] T fd0a6391bc474ee5aaf39c6589054b5f P 39f50f5b25d34bd188b4c6dced28aceb [term 1 FOLLOWER]: Refusing update from remote peer b7e22386798041d695b3f4cbcd82f494: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:35:05.798460 21099 raft_consensus.cc:1261] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Refusing update from remote peer b7e22386798041d695b3f4cbcd82f494: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:35:05.799022 21417 consensus_queue.cc:1035] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f50f5b25d34bd188b4c6dced28aceb" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 33173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:35:05.799278 21486 consensus_queue.cc:1035] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ff8eef27064232b25342734abbc47c" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:35:05.849844 20661 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:35:05.850082 20661 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:35:05.852686 20661 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3471e58ba77f49bfb97c69122ee8d00d: Sending DeleteTablet for 3 replicas of tablet fd0a6391bc474ee5aaf39c6589054b5f
I20260807 08:35:05.853372 20859 tablet_service.cc:1514] Processing DeleteTablet for tablet fd0a6391bc474ee5aaf39c6589054b5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:05 UTC) from {username='oryx', principal='oryx/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:41762
I20260807 08:35:05.853607 21494 tablet_replica.cc:330] T fd0a6391bc474ee5aaf39c6589054b5f P 39f50f5b25d34bd188b4c6dced28aceb: stopping tablet replica
I20260807 08:35:05.853740 21494 raft_consensus.cc:2229] T fd0a6391bc474ee5aaf39c6589054b5f P 39f50f5b25d34bd188b4c6dced28aceb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:35:05.853821 21060 tablet_service.cc:1514] Processing DeleteTablet for tablet fd0a6391bc474ee5aaf39c6589054b5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:05 UTC) from {username='oryx', principal='oryx/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:49690
I20260807 08:35:05.853891 21494 raft_consensus.cc:2258] T fd0a6391bc474ee5aaf39c6589054b5f P 39f50f5b25d34bd188b4c6dced28aceb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:05.854056 21443 tablet_replica.cc:330] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c: stopping tablet replica
I20260807 08:35:05.854189 21443 raft_consensus.cc:2229] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:35:05.854322 21443 raft_consensus.cc:2258] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:05.854732 21264 tablet_service.cc:1514] Processing DeleteTablet for tablet fd0a6391bc474ee5aaf39c6589054b5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:05 UTC) from {username='oryx', principal='oryx/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:56768
I20260807 08:35:05.855093 21495 tablet_replica.cc:330] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494: stopping tablet replica
I20260807 08:35:05.855248 21495 raft_consensus.cc:2229] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:35:05.855288 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 20773
I20260807 08:35:05.855512 21495 raft_consensus.cc:2258] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:05.864748 21443 ts_tablet_manager.cc:1905] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:35:05.868266 21495 ts_tablet_manager.cc:1905] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:35:05.869251 21443 ts_tablet_manager.cc:1918] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:35:05.869390 21443 log.cc:1196] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/fd0a6391bc474ee5aaf39c6589054b5f
I20260807 08:35:05.869726 21443 ts_tablet_manager.cc:1939] T fd0a6391bc474ee5aaf39c6589054b5f P a8ff8eef27064232b25342734abbc47c: Deleting consensus metadata
I20260807 08:35:05.871080 20644 catalog_manager.cc:4841] TS a8ff8eef27064232b25342734abbc47c (127.9.174.2:44939): tablet fd0a6391bc474ee5aaf39c6589054b5f (table test-table [id=a01bf592bd7a4a6583e64386b1e388fb]) successfully deleted
I20260807 08:35:05.874567 21495 ts_tablet_manager.cc:1918] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:35:05.874738 21495 log.cc:1196] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/fd0a6391bc474ee5aaf39c6589054b5f
I20260807 08:35:05.875144 21495 ts_tablet_manager.cc:1939] T fd0a6391bc474ee5aaf39c6589054b5f P b7e22386798041d695b3f4cbcd82f494: Deleting consensus metadata
I20260807 08:35:05.876567 20644 catalog_manager.cc:4841] TS b7e22386798041d695b3f4cbcd82f494 (127.9.174.3:36969): tablet fd0a6391bc474ee5aaf39c6589054b5f (table test-table [id=a01bf592bd7a4a6583e64386b1e388fb]) successfully deleted
W20260807 08:35:05.886301 20642 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:33173 (error 108)
I20260807 08:35:05.886714 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 20974
I20260807 08:35:05.896819 21541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:05.896045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:33173 (local address 127.0.0.1:41782)
0807 08:35:05.896483 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:05.896622 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:33173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":115,"threads_started":1}
W20260807 08:35:05.897149 20642 catalog_manager.cc:4584] TS 39f50f5b25d34bd188b4c6dced28aceb (127.9.174.1:33173): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fd0a6391bc474ee5aaf39c6589054b5f: Network error: Client connection negotiation failed: client connection to 127.9.174.1:33173: connect: Connection refused (error 111)
I20260807 08:35:05.905117 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 21164
I20260807 08:35:05.924165 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 20622
2026-08-07T08:35:05Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (4165 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:35:06 dist-test-slave-h3ft krb5kdc[21548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:35:06 dist-test-slave-h3ft krb5kdc[21548](info): set up 2 sockets
Aug 07 08:35:06 dist-test-slave-h3ft krb5kdc[21548](info): commencing operation
krb5kdc: starting...
W20260807 08:35:07.251771 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:35:07 dist-test-slave-h3ft krb5kdc[21548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:35:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:35:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:35:07.420269 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:34791
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:39281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:34791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:35:07.567581 21565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:07.568109 21565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:07.568199 21565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:07.573213 21565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:35:07.573367 21565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:07.573406 21565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:07.573441 21565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:35:07.573477 21565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:35:07.580531 21565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:39281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:34791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:34791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:07.582259 21565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:07.583732 21565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:07.595232 21576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:07.595424 21565 server_base.cc:1034] running on GCE node
W20260807 08:35:07.595232 21574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:07.595669 21573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:07.596553 21565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:07.598240 21565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:07.599464 21565 hybrid_clock.cc:648] HybridClock initialized: now 1786091707599442 us; error 45 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T08:35:07Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1684 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:35:07 dist-test-slave-h3ft krb5kdc[21582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:35:07 dist-test-slave-h3ft krb5kdc[21582](info): set up 2 sockets
Aug 07 08:35:07 dist-test-slave-h3ft krb5kdc[21582](info): commencing operation
krb5kdc: starting...
W20260807 08:35:08.934266 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.000s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:35:09 dist-test-slave-h3ft krb5kdc[21582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:35:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:35:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:35:09.095760 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:37703
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:37703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:35:09.228989 21600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:09.229295 21600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:09.229341 21600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:09.233546 21600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:35:09.233625 21600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:09.233646 21600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:09.233664 21600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:35:09.233717 21600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:35:09.240084 21600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:37703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:37703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:09.241245 21600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:09.242350 21600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:09.252076 21612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:09.252040 21605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:09.252107 21608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:09.252357 21600 server_base.cc:1034] running on GCE node
I20260807 08:35:09.253387 21600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:09.254765 21600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:09.256026 21600 hybrid_clock.cc:648] HybridClock initialized: now 1786091709256002 us; error 43 us; skew 500 ppm
Aug 07 08:35:09 dist-test-slave-h3ft krb5kdc[21582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091709, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:35:09.263943 21600 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:35:09.265358 21600 webserver.cc:460] Webserver started at http://127.9.174.62:34009/ using document root <none> and password file <none>
I20260807 08:35:09.266067 21600 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:09.266170 21600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:09.266445 21600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:09.268836 21600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "352a84be34144ac79af147211e1de784"
format_stamp: "Formatted at 2026-08-07 08:35:09 on dist-test-slave-h3ft"
server_key: "fdaae91314168259523747653f9c77e4"
server_key_iv: "f03017e063e3e2705c71d3488bca5d68"
server_key_version: "encryptionkey@0"
I20260807 08:35:09.269464 21600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "352a84be34144ac79af147211e1de784"
format_stamp: "Formatted at 2026-08-07 08:35:09 on dist-test-slave-h3ft"
server_key: "fdaae91314168259523747653f9c77e4"
server_key_iv: "f03017e063e3e2705c71d3488bca5d68"
server_key_version: "encryptionkey@0"
I20260807 08:35:09.274222 21600 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 08:35:09.277347 21622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:09.279237 21600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:35:09.279472 21600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "352a84be34144ac79af147211e1de784"
format_stamp: "Formatted at 2026-08-07 08:35:09 on dist-test-slave-h3ft"
server_key: "fdaae91314168259523747653f9c77e4"
server_key_iv: "f03017e063e3e2705c71d3488bca5d68"
server_key_version: "encryptionkey@0"
I20260807 08:35:09.279642 21600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:09.296403 21600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:09.300633 21600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:09.300854 21600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:09.310490 21600 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:37703
I20260807 08:35:09.310684 21701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:37703 every 8 connection(s)
I20260807 08:35:09.312002 21600 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:35:09.315685 21703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:09.317286 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 21600
I20260807 08:35:09.317476 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:35:09.317855 9912 external_mini_cluster.cc:1428] Setting key d780c3393e3ca873781d6d4f15b65dce
I20260807 08:35:09.323724 21703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784: Bootstrap starting.
I20260807 08:35:09.326565 21703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:09.327543 21703 log.cc:824] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:35:09 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:35:09.329806 21703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784: No bootstrap required, opened a new log
I20260807 08:35:09.334146 21703 raft_consensus.cc:348] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352a84be34144ac79af147211e1de784" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37703 } }
I20260807 08:35:09.334427 21703 raft_consensus.cc:374] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:09.334501 21703 raft_consensus.cc:729] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 352a84be34144ac79af147211e1de784, State: Initialized, Role: FOLLOWER
I20260807 08:35:09.335201 21703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352a84be34144ac79af147211e1de784" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37703 } }
I20260807 08:35:09.335424 21703 raft_consensus.cc:388] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:35:09.335515 21703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:35:09.335714 21703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:09.337191 21703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352a84be34144ac79af147211e1de784" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37703 } }
I20260807 08:35:09.337680 21703 leader_election.cc:302] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 352a84be34144ac79af147211e1de784; no voters:
I20260807 08:35:09.338132 21703 leader_election.cc:288] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:35:09.338379 21711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:35:09.338752 21711 raft_consensus.cc:686] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 1 LEADER]: Becoming Leader. State: Replica: 352a84be34144ac79af147211e1de784, State: Running, Role: LEADER
I20260807 08:35:09.339120 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:09.319780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42128 (local address 127.9.174.62:37703)
0807 08:35:09.320360 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:09.320408 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:09.320457 (+ 49us) server_negotiation.cc:407] Connection header received
0807 08:35:09.321350 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:09.321379 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:09.321821 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:09.322207 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:09.323409 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:09.324636 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:09.326008 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:09.326608 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:09.326848 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:09.330633 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:09.330641 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:09.330663 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:09.330733 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:09.334173 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:09.335209 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:09.335219 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:09.335230 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:09.335468 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:09.335984 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:09.335991 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:09.335996 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:09.336676 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:09.336945 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:09.337399 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 08:35:09.337838 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":185,"threads_started":1}
I20260807 08:35:09.339278 21711 consensus_queue.cc:237] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352a84be34144ac79af147211e1de784" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37703 } }
I20260807 08:35:09.339771 21703 sys_catalog.cc:564] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:35:09.341740 21715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 352a84be34144ac79af147211e1de784. Latest consensus state: current_term: 1 leader_uuid: "352a84be34144ac79af147211e1de784" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352a84be34144ac79af147211e1de784" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37703 } } }
I20260807 08:35:09.341920 21715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [sys.catalog]: This master's current role is: LEADER
I20260807 08:35:09.342293 21712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "352a84be34144ac79af147211e1de784" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352a84be34144ac79af147211e1de784" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37703 } } }
I20260807 08:35:09.342403 21712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [sys.catalog]: This master's current role is: LEADER
I20260807 08:35:09.342729 21718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:35:09.346863 21718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:35:09.353873 21718 catalog_manager.cc:1347] Generated new cluster ID: bb86423971ae4085ae87b5d042f34ea8
I20260807 08:35:09.354054 21718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:35:09.362676 21718 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:35:09.364530 21718 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:35:09.381165 21718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784: Generated new TSK 0
I20260807 08:35:09.382162 21718 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:35:09.428323 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37703
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:35:09.570062 21739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:09.570493 21739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:09.570549 21739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:09.575176 21739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:09.575264 21739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:09.575332 21739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:35:09.581431 21739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:09.582863 21739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:09.584228 21739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:09.596576 21751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:09.596611 21755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:09.596592 21752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:09.597045 21739 server_base.cc:1034] running on GCE node
I20260807 08:35:09.597622 21739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:09.598410 21739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:09.599627 21739 hybrid_clock.cc:648] HybridClock initialized: now 1786091709599575 us; error 68 us; skew 500 ppm
Aug 07 08:35:09 dist-test-slave-h3ft krb5kdc[21582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091709, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:35:09.603541 21739 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:35:09.604835 21739 webserver.cc:460] Webserver started at http://127.9.174.1:34147/ using document root <none> and password file <none>
I20260807 08:35:09.605613 21739 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:09.605733 21739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:09.606050 21739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:09.609012 21739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "db732df679f94425ad5bed51ae2d3af1"
format_stamp: "Formatted at 2026-08-07 08:35:09 on dist-test-slave-h3ft"
server_key: "9a4b39934a68e7fcd94eecdded007b66"
server_key_iv: "6f480f368753403d76d09f6c9fbfe235"
server_key_version: "encryptionkey@0"
I20260807 08:35:09.609797 21739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "db732df679f94425ad5bed51ae2d3af1"
format_stamp: "Formatted at 2026-08-07 08:35:09 on dist-test-slave-h3ft"
server_key: "9a4b39934a68e7fcd94eecdded007b66"
server_key_iv: "6f480f368753403d76d09f6c9fbfe235"
server_key_version: "encryptionkey@0"
I20260807 08:35:09.615226 21739 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 08:35:09.618664 21767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:09.620393 21739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:35:09.620584 21739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "db732df679f94425ad5bed51ae2d3af1"
format_stamp: "Formatted at 2026-08-07 08:35:09 on dist-test-slave-h3ft"
server_key: "9a4b39934a68e7fcd94eecdded007b66"
server_key_iv: "6f480f368753403d76d09f6c9fbfe235"
server_key_version: "encryptionkey@0"
I20260807 08:35:09.620764 21739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:09.646384 21739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:09.650893 21739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:09.651196 21739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:09.652034 21739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:35:09.653124 21739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:09.653177 21739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:09.653261 21739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:09.653297 21739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:09.665916 21739 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:41363
I20260807 08:35:09.666049 21932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:41363 every 8 connection(s)
I20260807 08:35:09.667296 21739 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:35:09.669831 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 21739
I20260807 08:35:09.670003 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:35:09.670394 9912 external_mini_cluster.cc:1428] Setting key b06113b96042cdd6f364c6f7c72a514c
Aug 07 08:35:09 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091709, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
I20260807 08:35:09.689929 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:09.671948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:52949 (local address 127.9.174.62:37703)
0807 08:35:09.672163 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:09.672169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:09.673398 (+ 1229us) server_negotiation.cc:407] Connection header received
0807 08:35:09.675037 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:09.675044 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:09.675144 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:09.675290 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:09.677425 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:09.678031 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:09.680322 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:09.680596 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:09.680713 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:09.684722 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:09.684728 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:09.684731 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:09.684777 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:09.686757 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:09.687461 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:09.687463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:09.687465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:09.687659 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:09.688091 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:09.688094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:09.688096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:09.688435 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:09.688568 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:09.689546 (+ 978us) server_negotiation.cc:299] Negotiation successful
0807 08:35:09.689780 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 08:35:09.691474 21938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:09.672508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:52949)
0807 08:35:09.673178 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:09.673242 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:09.674730 (+ 1488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:09.675595 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:09.675606 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:09.676300 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:09.677192 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:09.677219 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:09.678230 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:09.678237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:09.680105 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:09.680121 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:09.680838 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:09.680846 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:09.681127 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:09.681980 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:09.682007 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:09.684424 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:09.686910 (+ 2486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:09.686916 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:09.686932 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:09.687341 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:09.687802 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:09.687805 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:09.687806 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:09.687975 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:09.688603 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:09.688613 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:09.689063 (+ 450us) client_negotiation.cc:769] Sending connection context
0807 08:35:09.689367 (+ 304us) client_negotiation.cc:240] Negotiation successful
0807 08:35:09.689904 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":211,"threads_started":1}
I20260807 08:35:09.693343 21936 heartbeater.cc:347] Connected to a master server at 127.9.174.62:37703
I20260807 08:35:09.693769 21936 heartbeater.cc:464] Registering TS with master...
I20260807 08:35:09.694587 21936 heartbeater.cc:511] Master 127.9.174.62:37703 requested a full tablet report, sending...
I20260807 08:35:09.696874 21647 ts_manager.cc:194] Registered new tserver with Master: db732df679f94425ad5bed51ae2d3af1 (127.9.174.1:41363)
I20260807 08:35:09.698949 21647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:52949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:35:09.711371 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37703
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:35:09.839612 21944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:09.840016 21944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:09.840066 21944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:09.844753 21944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:09.844838 21944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:09.844900 21944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:35:09.851181 21944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:09.852366 21944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:09.853510 21944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:09.864908 21950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:09.865188 21944 server_base.cc:1034] running on GCE node
W20260807 08:35:09.864908 21953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:09.864888 21951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:09.866009 21944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:09.867069 21944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:09.868300 21944 hybrid_clock.cc:648] HybridClock initialized: now 1786091709868275 us; error 49 us; skew 500 ppm
Aug 07 08:35:09 dist-test-slave-h3ft krb5kdc[21582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091709, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:35:09.872397 21944 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:35:09.873721 21944 webserver.cc:460] Webserver started at http://127.9.174.2:40623/ using document root <none> and password file <none>
I20260807 08:35:09.874508 21944 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:09.874621 21944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:09.874914 21944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:09.877640 21944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "fe5fa9d4d1344fb5815df02b59c62b01"
format_stamp: "Formatted at 2026-08-07 08:35:09 on dist-test-slave-h3ft"
server_key: "13adf6391d4288f0140481f8017bba6e"
server_key_iv: "bc96b4d740324082408f2e411e93bbfd"
server_key_version: "encryptionkey@0"
I20260807 08:35:09.878350 21944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "fe5fa9d4d1344fb5815df02b59c62b01"
format_stamp: "Formatted at 2026-08-07 08:35:09 on dist-test-slave-h3ft"
server_key: "13adf6391d4288f0140481f8017bba6e"
server_key_iv: "bc96b4d740324082408f2e411e93bbfd"
server_key_version: "encryptionkey@0"
I20260807 08:35:09.883980 21944 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 08:35:09.887676 21963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:09.889317 21944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:35:09.889503 21944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "fe5fa9d4d1344fb5815df02b59c62b01"
format_stamp: "Formatted at 2026-08-07 08:35:09 on dist-test-slave-h3ft"
server_key: "13adf6391d4288f0140481f8017bba6e"
server_key_iv: "bc96b4d740324082408f2e411e93bbfd"
server_key_version: "encryptionkey@0"
I20260807 08:35:09.889649 21944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:09.902038 21944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:09.906016 21944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:09.906268 21944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:09.907097 21944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:35:09.908169 21944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:09.908255 21944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:09.908329 21944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:09.908381 21944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:09.921152 21944 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:36489
I20260807 08:35:09.921190 22119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:36489 every 8 connection(s)
I20260807 08:35:09.922335 21944 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:35:09.931952 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 21944
I20260807 08:35:09.932116 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:35:09.932557 9912 external_mini_cluster.cc:1428] Setting key 3987dc133768a2da3e2eabd22b519044
Aug 07 08:35:09 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091709, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:35:09.940316 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:09.924908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:51407 (local address 127.9.174.62:37703)
0807 08:35:09.925108 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:09.925113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:09.926298 (+ 1185us) server_negotiation.cc:407] Connection header received
0807 08:35:09.928226 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:09.928229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:09.928298 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:09.928430 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:09.929866 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:09.930468 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:09.931959 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:09.932365 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:09.932501 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:09.935988 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:09.935993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:09.935996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:09.936033 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:09.937810 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:09.938397 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:09.938401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:09.938402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:09.938486 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:09.938918 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:09.938921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:09.938923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:09.939202 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:09.939320 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:09.940000 (+ 680us) server_negotiation.cc:299] Negotiation successful
0807 08:35:09.940139 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 08:35:09.941318 22124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:09.925389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:51407)
0807 08:35:09.926054 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:09.926118 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:09.927965 (+ 1847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:09.928655 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:09.928662 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:09.929110 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:09.929683 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:09.929698 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:09.930646 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:09.930650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:09.931804 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:09.931815 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:09.932561 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:09.932566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:09.932851 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:09.933687 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:09.933713 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:09.935778 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:09.937962 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:09.937969 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:09.937987 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:09.938275 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:09.938644 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:09.938647 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:09.938656 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:09.938804 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:09.939317 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:09.939325 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:09.939664 (+ 339us) client_negotiation.cc:769] Sending connection context
0807 08:35:09.939879 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 08:35:09.940242 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":158,"threads_started":1}
I20260807 08:35:09.942934 22120 heartbeater.cc:347] Connected to a master server at 127.9.174.62:37703
I20260807 08:35:09.943243 22120 heartbeater.cc:464] Registering TS with master...
I20260807 08:35:09.943813 22120 heartbeater.cc:511] Master 127.9.174.62:37703 requested a full tablet report, sending...
I20260807 08:35:09.945292 21647 ts_manager.cc:194] Registered new tserver with Master: fe5fa9d4d1344fb5815df02b59c62b01 (127.9.174.2:36489)
I20260807 08:35:09.946007 21647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:51407
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:35:09.976565 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37703
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:35:10.104346 22129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:10.104786 22129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:10.104918 22129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:10.109321 22129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:10.109402 22129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:10.109464 22129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:35:10.115310 22129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:10.116448 22129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:10.117489 22129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:10.128742 22142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:10.128743 22138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:10.128788 22139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:10.129302 22129 server_base.cc:1034] running on GCE node
I20260807 08:35:10.129809 22129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:10.130535 22129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:10.131994 22129 hybrid_clock.cc:648] HybridClock initialized: now 1786091710131720 us; error 85 us; skew 500 ppm
Aug 07 08:35:10 dist-test-slave-h3ft krb5kdc[21582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091710, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:35:10.136013 22129 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:35:10.137332 22129 webserver.cc:460] Webserver started at http://127.9.174.3:46119/ using document root <none> and password file <none>
I20260807 08:35:10.138140 22129 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:10.138221 22129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:10.138520 22129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:10.141594 22129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "72740aea11a44edc852f82c91ee5e464"
format_stamp: "Formatted at 2026-08-07 08:35:10 on dist-test-slave-h3ft"
server_key: "da2b52b78b2b5e12706cd3049ed4d904"
server_key_iv: "71c554c41a02f3d6a8962dedcc41bfc5"
server_key_version: "encryptionkey@0"
I20260807 08:35:10.142392 22129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "72740aea11a44edc852f82c91ee5e464"
format_stamp: "Formatted at 2026-08-07 08:35:10 on dist-test-slave-h3ft"
server_key: "da2b52b78b2b5e12706cd3049ed4d904"
server_key_iv: "71c554c41a02f3d6a8962dedcc41bfc5"
server_key_version: "encryptionkey@0"
I20260807 08:35:10.148176 22129 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 08:35:10.151751 22155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:10.153473 22129 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:35:10.153609 22129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "72740aea11a44edc852f82c91ee5e464"
format_stamp: "Formatted at 2026-08-07 08:35:10 on dist-test-slave-h3ft"
server_key: "da2b52b78b2b5e12706cd3049ed4d904"
server_key_iv: "71c554c41a02f3d6a8962dedcc41bfc5"
server_key_version: "encryptionkey@0"
I20260807 08:35:10.153723 22129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:10.169142 22129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:10.173288 22129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:10.173542 22129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:10.174371 22129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:35:10.175446 22129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:10.175515 22129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:10.175565 22129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:10.175637 22129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:10.189227 22129 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:35755
I20260807 08:35:10.189257 22324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:35755 every 8 connection(s)
I20260807 08:35:10.190577 22129 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:35:10.196966 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 22129
I20260807 08:35:10.197170 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:35:10.197570 9912 external_mini_cluster.cc:1428] Setting key f001789da10174385a46f92eb4fef32e
Aug 07 08:35:10 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091710, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:35:10.208938 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:10.193296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:56697 (local address 127.9.174.62:37703)
0807 08:35:10.193532 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:10.193538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:10.194756 (+ 1218us) server_negotiation.cc:407] Connection header received
0807 08:35:10.196564 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:10.196568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:10.196639 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:10.196750 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:10.198621 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:10.199440 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:10.200774 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:10.201018 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:10.201159 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:10.204519 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:10.204522 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:10.204524 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:10.204551 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:10.206212 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:10.206916 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:10.206918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:10.206919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:10.207057 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:10.207599 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:10.207602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:10.207603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:10.207857 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:10.207960 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:10.208641 (+ 681us) server_negotiation.cc:299] Negotiation successful
0807 08:35:10.208792 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 08:35:10.210170 22330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:10.193934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:56697)
0807 08:35:10.194540 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:10.194597 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:10.196272 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:10.197055 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:10.197068 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:10.197722 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:10.198400 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:10.198416 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:10.199599 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:10.199609 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:10.200605 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:10.200615 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:10.201220 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:10.201226 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:10.201417 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:10.202196 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:10.202215 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:10.204266 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:10.206379 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:10.206385 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:10.206403 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:10.206791 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:10.207256 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:10.207259 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:10.207261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:10.207452 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:10.207988 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:10.207995 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:10.208347 (+ 352us) client_negotiation.cc:769] Sending connection context
0807 08:35:10.208564 (+ 217us) client_negotiation.cc:240] Negotiation successful
0807 08:35:10.208970 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":147,"threads_started":1}
I20260807 08:35:10.212009 22326 heartbeater.cc:347] Connected to a master server at 127.9.174.62:37703
I20260807 08:35:10.212357 22326 heartbeater.cc:464] Registering TS with master...
I20260807 08:35:10.213153 22326 heartbeater.cc:511] Master 127.9.174.62:37703 requested a full tablet report, sending...
I20260807 08:35:10.214506 21647 ts_manager.cc:194] Registered new tserver with Master: 72740aea11a44edc852f82c91ee5e464 (127.9.174.3:35755)
I20260807 08:35:10.215286 21647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:56697
I20260807 08:35:10.225476 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.1.keytab.
Entry for principal oryx/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.1.keytab.
WARNING: no policy specified for oryx/127.9.174.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.174.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.2.keytab.
Entry for principal oryx/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.2.keytab.
WARNING: no policy specified for oryx/127.9.174.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.174.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.3.keytab.
Entry for principal oryx/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.3.keytab.
I20260807 08:35:10.331600 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 21739
I20260807 08:35:10.348421 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 21944
I20260807 08:35:10.370494 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 22129
I20260807 08:35:10.386111 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 21600
I20260807 08:35:10.403322 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:37703
--webserver_interface=127.9.174.62
--webserver_port=34009
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:37703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:35:10.524236 22341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:10.524526 22341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:10.524567 22341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:10.528779 22341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:35:10.528859 22341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:10.528878 22341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:10.528895 22341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:35:10.528910 22341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:35:10.535310 22341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:37703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:37703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=34009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:10.536537 22341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:10.537647 22341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:10.547456 22348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:10.547482 22351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:10.547487 22347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:10.548056 22341 server_base.cc:1034] running on GCE node
I20260807 08:35:10.548700 22341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:10.549880 22341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:10.551106 22341 hybrid_clock.cc:648] HybridClock initialized: now 1786091710551062 us; error 61 us; skew 500 ppm
Aug 07 08:35:10 dist-test-slave-h3ft krb5kdc[21582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091710, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:35:10.554436 22341 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:35:10.555606 22341 webserver.cc:460] Webserver started at http://127.9.174.62:34009/ using document root <none> and password file <none>
I20260807 08:35:10.556259 22341 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:10.556383 22341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:10.560791 22341 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:35:10.563692 22362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:10.565289 22341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:35:10.565470 22341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "352a84be34144ac79af147211e1de784"
format_stamp: "Formatted at 2026-08-07 08:35:09 on dist-test-slave-h3ft"
server_key: "fdaae91314168259523747653f9c77e4"
server_key_iv: "f03017e063e3e2705c71d3488bca5d68"
server_key_version: "encryptionkey@0"
I20260807 08:35:10.566102 22341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:10.586411 22341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:10.590049 22341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:10.590294 22341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:10.599123 22341 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:37703
I20260807 08:35:10.599238 22439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:37703 every 8 connection(s)
I20260807 08:35:10.600368 22341 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:35:10.603679 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 22341
I20260807 08:35:10.604206 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:41363
--local_ip_for_outbound_sockets=127.9.174.1
--tserver_master_addrs=127.9.174.62:37703
--webserver_port=34147
--webserver_interface=127.9.174.1
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.174.1
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.1.keytab with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:35:10.606182 22441 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:35:10.608534 22441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784: Bootstrap starting.
I20260807 08:35:10.632257 22441 log.cc:824] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784: Log is configured to *not* fsync() on all Append() calls
I20260807 08:35:10.640847 22441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:35:10.641417 22441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784: Bootstrap complete.
I20260807 08:35:10.646896 22441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352a84be34144ac79af147211e1de784" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37703 } }
I20260807 08:35:10.647569 22441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 352a84be34144ac79af147211e1de784, State: Initialized, Role: FOLLOWER
I20260807 08:35:10.648247 22441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352a84be34144ac79af147211e1de784" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37703 } }
I20260807 08:35:10.648432 22441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:35:10.648532 22441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:35:10.648670 22441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:35:10.651324 22441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352a84be34144ac79af147211e1de784" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37703 } }
I20260807 08:35:10.651748 22441 leader_election.cc:302] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 352a84be34144ac79af147211e1de784; no voters:
I20260807 08:35:10.652191 22441 leader_election.cc:288] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 08:35:10.652310 22448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:35:10.652530 22448 raft_consensus.cc:686] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [term 2 LEADER]: Becoming Leader. State: Replica: 352a84be34144ac79af147211e1de784, State: Running, Role: LEADER
I20260807 08:35:10.652937 22448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352a84be34144ac79af147211e1de784" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37703 } }
I20260807 08:35:10.653893 22441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:35:10.654640 22449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "352a84be34144ac79af147211e1de784" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352a84be34144ac79af147211e1de784" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37703 } } }
I20260807 08:35:10.654881 22449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [sys.catalog]: This master's current role is: LEADER
I20260807 08:35:10.655758 22459 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:35:10.655928 22451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 352a84be34144ac79af147211e1de784. Latest consensus state: current_term: 2 leader_uuid: "352a84be34144ac79af147211e1de784" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352a84be34144ac79af147211e1de784" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37703 } } }
I20260807 08:35:10.656075 22451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784 [sys.catalog]: This master's current role is: LEADER
I20260807 08:35:10.659902 22459 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:35:10.660686 22459 catalog_manager.cc:1259] Loaded cluster ID: bb86423971ae4085ae87b5d042f34ea8
I20260807 08:35:10.660755 22459 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:35:10.663264 22459 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:35:10.663980 22459 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 352a84be34144ac79af147211e1de784: Loaded TSK: 0
I20260807 08:35:10.665340 22459 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:35:10.734411 22446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:10.734728 22446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:10.734776 22446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:10.739018 22446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:10.739099 22446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:10.739161 22446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:35:10.744968 22446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.1.keytab
--principal=oryx/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:41363
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=34147
--tserver_master_addrs=127.9.174.62:37703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:10.746145 22446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:10.747354 22446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:10.758111 22489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:10.758098 22490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:10.758599 22446 server_base.cc:1034] running on GCE node
W20260807 08:35:10.758783 22492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:10.759320 22446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:10.760028 22446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:10.761222 22446 hybrid_clock.cc:648] HybridClock initialized: now 1786091710761187 us; error 57 us; skew 500 ppm
Aug 07 08:35:10 dist-test-slave-h3ft krb5kdc[21582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091710, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:35:10.764694 22446 init.cc:377] Logged in from keytab as oryx/127.9.174.1@KRBTEST.COM (short username oryx)
I20260807 08:35:10.765745 22446 webserver.cc:460] Webserver started at http://127.9.174.1:34147/ using document root <none> and password file <none>
I20260807 08:35:10.766372 22446 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:10.766500 22446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:10.770969 22446 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:35:10.773623 22499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:10.775441 22446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:35:10.775614 22446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "db732df679f94425ad5bed51ae2d3af1"
format_stamp: "Formatted at 2026-08-07 08:35:09 on dist-test-slave-h3ft"
server_key: "9a4b39934a68e7fcd94eecdded007b66"
server_key_iv: "6f480f368753403d76d09f6c9fbfe235"
server_key_version: "encryptionkey@0"
I20260807 08:35:10.776259 22446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:10.799613 22446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:10.803479 22446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:10.803730 22446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:10.804438 22446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:35:10.805454 22446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:10.805533 22446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:10.805611 22446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:10.805660 22446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:10.818084 22446 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:41363
I20260807 08:35:10.818181 22683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:41363 every 8 connection(s)
I20260807 08:35:10.819356 22446 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:35:10.823632 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 22446
I20260807 08:35:10.824219 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:36489
--local_ip_for_outbound_sockets=127.9.174.2
--tserver_master_addrs=127.9.174.62:37703
--webserver_port=40623
--webserver_interface=127.9.174.2
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.174.2
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.2.keytab with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:35:10 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:10 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:10.838217 22687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:10.822457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:59861)
0807 08:35:10.823063 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:10.823123 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:10.824696 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:10.826562 (+ 1866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:10.826576 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:10.827279 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:10.828059 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:10.828087 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:10.830406 (+ 2319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:10.830409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:10.831659 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:10.831673 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:10.833139 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:10.833144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:10.833321 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:10.834046 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:10.834064 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:10.837285 (+ 3221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":178,"threads_started":1}
W20260807 08:35:10.839073 22684 heartbeater.cc:644] Failed to heartbeat to 127.9.174.62:37703 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.174.62:37703: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
W20260807 08:35:10.839430 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:10.822410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:59861 (local address 127.9.174.62:37703)
0807 08:35:10.823115 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:10.823136 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:10.823307 (+ 171us) server_negotiation.cc:407] Connection header received
0807 08:35:10.825314 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:10.825341 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:10.825893 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:10.826418 (+ 525us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:10.828324 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:10.830183 (+ 1859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:10.831845 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:10.832906 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:10.833186 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:10.837813 (+ 4627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:59861: BlockingRecv error: recv got EOF from 127.9.174.1:59861 (error 108)
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":225,"threads_started":1}
Aug 07 08:35:10 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:10 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:10.857184 22687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:10.839791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:59843)
0807 08:35:10.840148 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:10.840170 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:10.840304 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:10.840649 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:10.840652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:10.840882 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:10.841021 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:10.841027 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:10.842051 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:10.842055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:10.843246 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:10.843261 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:10.853456 (+ 10195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:10.853462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:10.853621 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:10.854310 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:10.854329 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:10.857018 (+ 2689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233}
W20260807 08:35:10.857659 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:10.840121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:59843 (local address 127.9.174.62:37703)
0807 08:35:10.840291 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:10.840296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:10.840314 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:35:10.840451 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:10.840453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:10.840509 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:10.840623 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:10.841188 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:10.841820 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:10.843475 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:10.844104 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:10.844269 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:10.857453 (+ 13184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:59843: BlockingRecv error: recv got EOF from 127.9.174.1:59843 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 07 08:35:10 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:10 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:10.866456 22687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:10.858163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:36775)
0807 08:35:10.858322 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:10.858341 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:10.858475 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:10.858936 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:10.858940 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:10.859181 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:10.859306 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:10.859313 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:10.860323 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:10.860341 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:10.861138 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:10.861148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:10.861704 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:10.861707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:10.861803 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:10.862479 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:10.862501 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:10.866229 (+ 3728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 08:35:10.866598 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:10.858224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:36775 (local address 127.9.174.62:37703)
0807 08:35:10.858647 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:10.858672 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:10.858696 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:35:10.858748 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:10.858751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:10.858805 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:10.858910 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:10.859423 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:10.860197 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:10.861257 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:10.861579 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:10.861697 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:10.866445 (+ 4748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:36775: BlockingRecv error: recv got EOF from 127.9.174.1:36775 (error 108)
Metrics: {"server-negotiator.queue_time_us":325}
W20260807 08:35:10.867092 22684 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:35:10.959041 22689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:10.959406 22689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:10.959494 22689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:10.963657 22689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:10.963778 22689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:10.963888 22689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:35:10.969496 22689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.2.keytab
--principal=oryx/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:36489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=40623
--tserver_master_addrs=127.9.174.62:37703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:10.971028 22689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:10.972235 22689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:10.984155 22701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:10.984247 22708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:10.984155 22698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:10.984530 22689 server_base.cc:1034] running on GCE node
I20260807 08:35:10.985183 22689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:10.986091 22689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:10.987322 22689 hybrid_clock.cc:648] HybridClock initialized: now 1786091710987296 us; error 44 us; skew 500 ppm
Aug 07 08:35:10 dist-test-slave-h3ft krb5kdc[21582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091710, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:35:10.991308 22689 init.cc:377] Logged in from keytab as oryx/127.9.174.2@KRBTEST.COM (short username oryx)
I20260807 08:35:10.992628 22689 webserver.cc:460] Webserver started at http://127.9.174.2:40623/ using document root <none> and password file <none>
I20260807 08:35:10.993453 22689 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:10.993607 22689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:10.999421 22689 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:35:11.002405 22719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:11.004443 22689 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:35:11.004635 22689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "fe5fa9d4d1344fb5815df02b59c62b01"
format_stamp: "Formatted at 2026-08-07 08:35:09 on dist-test-slave-h3ft"
server_key: "13adf6391d4288f0140481f8017bba6e"
server_key_iv: "bc96b4d740324082408f2e411e93bbfd"
server_key_version: "encryptionkey@0"
I20260807 08:35:11.005451 22689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:11.027101 22689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:11.031512 22689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:11.031803 22689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:11.032670 22689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:35:11.033839 22689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:11.033932 22689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:11.034014 22689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:11.034065 22689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:11.046207 22689 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:36489
I20260807 08:35:11.046227 22885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:36489 every 8 connection(s)
I20260807 08:35:11.047748 22689 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:35:11.056162 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 22689
I20260807 08:35:11.056803 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:35755
--local_ip_for_outbound_sockets=127.9.174.3
--tserver_master_addrs=127.9.174.62:37703
--webserver_port=46119
--webserver_interface=127.9.174.3
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.174.3
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.3.keytab with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:35:11 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:11 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 08:35:11.063647 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:11.050510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:41677 (local address 127.9.174.62:37703)
0807 08:35:11.050762 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:11.050768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:11.052104 (+ 1336us) server_negotiation.cc:407] Connection header received
0807 08:35:11.053817 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:11.053821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:11.053967 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:11.054102 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:11.055980 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.056680 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.058125 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.058359 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.058499 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:11.063412 (+ 4913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:41677: BlockingRecv error: recv got EOF from 127.9.174.2:41677 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 08:35:11.064397 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:11.051150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:41677)
0807 08:35:11.051850 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:11.051933 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:11.053534 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:11.054405 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:11.054419 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:11.054960 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:11.055788 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:11.055807 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.056836 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.056839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:11.057978 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:11.057990 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.058565 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.058569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:11.058777 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:11.059712 (+ 935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:11.059742 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:11.063190 (+ 3448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":206,"threads_started":1}
W20260807 08:35:11.065238 22886 heartbeater.cc:644] Failed to heartbeat to 127.9.174.62:37703 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.174.62:37703: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Aug 07 08:35:11 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:11 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:11.072659 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:11.066020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:39969)
0807 08:35:11.066171 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:11.066190 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:11.066305 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:11.066613 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:11.066616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:11.066840 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:11.066974 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:11.066979 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.067830 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.067835 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:11.068429 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:11.068437 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.068923 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.068927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:11.069030 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:11.069703 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:11.069716 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:11.072425 (+ 2709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 08:35:11.072743 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:11.066181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:39969 (local address 127.9.174.62:37703)
0807 08:35:11.066317 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:11.066328 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:11.066350 (+ 22us) server_negotiation.cc:407] Connection header received
0807 08:35:11.066437 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:11.066440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:11.066487 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:11.066588 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:11.067131 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.067707 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.068569 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.068795 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.068913 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:11.072609 (+ 3696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:39969: BlockingRecv error: recv got EOF from 127.9.174.2:39969 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 08:35:11 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:11 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:11.080736 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:11.073740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:41363)
0807 08:35:11.073929 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:11.073954 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:11.074048 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:11.074399 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:11.074402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:11.074572 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:11.074688 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:11.074695 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.075844 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.075848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:11.076663 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:11.076676 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.077298 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.077301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:11.077376 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:11.078055 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:11.078070 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:11.080531 (+ 2461us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 08:35:11.080932 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:11.073810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:41363 (local address 127.9.174.62:37703)
0807 08:35:11.073956 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:11.073961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:11.074027 (+ 66us) server_negotiation.cc:407] Connection header received
0807 08:35:11.074150 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:11.074153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:11.074209 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:11.074307 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:11.074815 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.075708 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.076801 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.077162 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.077265 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:11.080739 (+ 3474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:41363: BlockingRecv error: recv got EOF from 127.9.174.2:41363 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 08:35:11.081267 22886 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:35:11.207788 22889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:11.208177 22889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:11.208230 22889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:11.213055 22889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:11.213171 22889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:11.213246 22889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:35:11.219770 22889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/krb5kdc/oryx_127.9.174.3.keytab
--principal=oryx/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:35755
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=46119
--tserver_master_addrs=127.9.174.62:37703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:11.221398 22889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:11.222813 22889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:11.235459 22898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:11.235543 22894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:11.235772 22895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:11.235716 22889 server_base.cc:1034] running on GCE node
I20260807 08:35:11.236539 22889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:11.237510 22889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:11.238701 22889 hybrid_clock.cc:648] HybridClock initialized: now 1786091711238659 us; error 56 us; skew 500 ppm
Aug 07 08:35:11 dist-test-slave-h3ft krb5kdc[21582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091711, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:35:11.242722 22889 init.cc:377] Logged in from keytab as oryx/127.9.174.3@KRBTEST.COM (short username oryx)
I20260807 08:35:11.244102 22889 webserver.cc:460] Webserver started at http://127.9.174.3:46119/ using document root <none> and password file <none>
I20260807 08:35:11.244832 22889 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:11.244925 22889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:11.250228 22889 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 08:35:11.253284 22912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:11.255223 22889 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:35:11.255385 22889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "72740aea11a44edc852f82c91ee5e464"
format_stamp: "Formatted at 2026-08-07 08:35:10 on dist-test-slave-h3ft"
server_key: "da2b52b78b2b5e12706cd3049ed4d904"
server_key_iv: "71c554c41a02f3d6a8962dedcc41bfc5"
server_key_version: "encryptionkey@0"
I20260807 08:35:11.256227 22889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:11.287649 22889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:11.292126 22889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:11.292419 22889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:11.293278 22889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:35:11.294373 22889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:11.294466 22889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:11.294548 22889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:11.294585 22889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:11.306773 22889 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:35755
I20260807 08:35:11.306829 23095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:35755 every 8 connection(s)
I20260807 08:35:11.308210 22889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:35:11.308775 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 22889
I20260807 08:35:11.320823 22688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:11.309855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42138 (local address 127.9.174.62:37703)
0807 08:35:11.310114 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:11.310119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:11.310335 (+ 216us) server_negotiation.cc:407] Connection header received
0807 08:35:11.310594 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:11.310600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:11.310664 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:11.310760 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:11.311869 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.312708 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.313683 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.313929 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.314061 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:11.315916 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:11.315924 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:11.315939 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:11.315969 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:11.318184 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:11.318727 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:11.318733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:11.318738 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:11.318946 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:11.319289 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:11.319292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:11.319294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:11.319847 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:11.320092 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:11.320319 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 08:35:11.320633 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
Aug 07 08:35:11 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:11 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 08:35:11.323885 23100 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:11.311264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:47945 (local address 127.9.174.62:37703)
0807 08:35:11.311573 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:11.311577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:11.312260 (+ 683us) server_negotiation.cc:407] Connection header received
0807 08:35:11.313865 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:11.313869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:11.313931 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:11.314044 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:11.315799 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.316447 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.318322 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.318738 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.318820 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:11.323441 (+ 4621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:47945: BlockingRecv error: recv got EOF from 127.9.174.3:47945 (error 108)
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":121,"threads_started":1}
I20260807 08:35:11.324394 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:11.311530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:47945)
0807 08:35:11.312097 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:11.312148 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:11.313630 (+ 1482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:11.314266 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:11.314276 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:11.314917 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:11.315624 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:11.315640 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.316608 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.316612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:11.318168 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:11.318179 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.318976 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.318983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:11.319274 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:11.319876 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:11.319902 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:11.323221 (+ 3319us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":174,"threads_started":1}
W20260807 08:35:11.325142 23096 heartbeater.cc:644] Failed to heartbeat to 127.9.174.62:37703 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.174.62:37703: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Aug 07 08:35:11 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:11 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:11.332438 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:11.325666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:60005)
0807 08:35:11.325810 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:11.325826 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:11.325924 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:11.326153 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:11.326155 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:11.326323 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:11.326443 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:11.326448 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.327706 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.327711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:11.328582 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:11.328594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.329108 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.329111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:11.329196 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:11.329775 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:11.329788 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:11.332282 (+ 2494us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 08:35:11.332772 23100 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:11.325770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:60005 (local address 127.9.174.62:37703)
0807 08:35:11.325945 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:11.325948 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:11.325961 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:35:11.325996 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:11.325998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:11.326047 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:11.326116 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:11.326587 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.327542 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.328712 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.328944 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.329037 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:11.332635 (+ 3598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:60005: BlockingRecv error: recv got EOF from 127.9.174.3:60005 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 08:35:11 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:11 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:11.339202 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:11.333266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:55049)
0807 08:35:11.333433 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:11.333447 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:11.333578 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:11.333882 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:11.333885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:11.334115 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:11.334210 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:11.334216 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.335052 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.335058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:11.335718 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:11.335728 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.336335 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.336338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:11.336443 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:11.337110 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:11.337127 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:11.339052 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":82}
W20260807 08:35:11.339318 23100 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:11.333387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:55049 (local address 127.9.174.62:37703)
0807 08:35:11.333614 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:11.333621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:11.333651 (+ 30us) server_negotiation.cc:407] Connection header received
0807 08:35:11.333708 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:11.333710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:11.333758 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:11.333860 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:11.334341 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.334901 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.335845 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.336212 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.336326 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:11.339193 (+ 2867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:55049: BlockingRecv error: recv got EOF from 127.9.174.3:55049 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
W20260807 08:35:11.339603 23096 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 08:35:11 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:11 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:11.875422 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:11.868096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:35573)
0807 08:35:11.868481 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:11.868497 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:11.868624 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:11.868932 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:11.868936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:11.869172 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:11.869321 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:11.869327 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.870318 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.870321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:11.871112 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:11.871123 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.871650 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.871653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:11.871732 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:11.872592 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:11.872609 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:11.875206 (+ 2597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":132,"threads_started":1}
W20260807 08:35:11.875680 23106 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:11.868163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:35573 (local address 127.9.174.62:37703)
0807 08:35:11.868534 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:11.868538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:11.868561 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:35:11.868727 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:11.868730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:11.868785 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:11.868870 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:11.869446 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.870151 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.871242 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:11.871498 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:11.871616 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:11.875491 (+ 3875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:35573: BlockingRecv error: recv got EOF from 127.9.174.1:35573 (error 108)
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":180,"threads_started":1}
Aug 07 08:35:12 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:12 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:12.090721 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:12.082331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:51019)
0807 08:35:12.082899 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:12.082917 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:12.083158 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:12.083560 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:12.083563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:12.083837 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:12.083959 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:12.083966 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:12.084855 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:12.084862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:12.085558 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:12.085568 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:12.086116 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:12.086120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:12.086208 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:12.087418 (+ 1210us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:12.087436 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:12.090496 (+ 3060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":121,"threads_started":1}
W20260807 08:35:12.090889 23106 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:12.082459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:51019 (local address 127.9.174.62:37703)
0807 08:35:12.082694 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:12.082701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:12.083057 (+ 356us) server_negotiation.cc:407] Connection header received
0807 08:35:12.083319 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:12.083323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:12.083390 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:12.083516 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:12.084113 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:12.084721 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:12.085721 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:12.085972 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:12.086103 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:12.090717 (+ 4614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:51019: BlockingRecv error: recv got EOF from 127.9.174.2:51019 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 07 08:35:12 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:12 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:12.349058 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:12.340457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:45371)
0807 08:35:12.341072 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:12.341091 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:12.341314 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:12.341721 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:12.341724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:12.342073 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:12.342213 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:12.342220 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:12.343310 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:12.343315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:12.344064 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:12.344074 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:12.344653 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:12.344657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:12.344753 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:12.345631 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:12.345649 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:12.348869 (+ 3220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":191,"threads_started":1}
W20260807 08:35:12.349325 23106 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:12.340696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:45371 (local address 127.9.174.62:37703)
0807 08:35:12.340982 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:12.340990 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:12.341222 (+ 232us) server_negotiation.cc:407] Connection header received
0807 08:35:12.341469 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:12.341475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:12.341553 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:12.341678 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:12.342410 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:12.343166 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:12.344214 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:12.344492 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:12.344638 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:12.349122 (+ 4484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:45371: BlockingRecv error: recv got EOF from 127.9.174.3:45371 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Aug 07 08:35:12 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:12 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:12.886610 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:12.876805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:48091)
0807 08:35:12.877176 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:12.877194 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:12.877327 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:12.877770 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:12.877775 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:12.878001 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:12.878119 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:12.878126 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:12.878961 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:12.878965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:12.879655 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:12.879665 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:12.880550 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:12.880553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:12.880622 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:12.883678 (+ 3056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:12.883691 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:12.886398 (+ 2707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":132,"threads_started":1}
W20260807 08:35:12.887132 23110 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:12.876914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:48091 (local address 127.9.174.62:37703)
0807 08:35:12.877272 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:12.877280 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:12.877303 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:35:12.877484 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:12.877488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:12.877572 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:12.877687 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:12.878265 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:12.878837 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:12.879869 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:12.880390 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:12.880552 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:12.886880 (+ 6328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:48091: BlockingRecv error: recv got EOF from 127.9.174.1:48091 (error 108)
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":126,"threads_started":1}
Aug 07 08:35:13 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:13 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:13.098920 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:13.092050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:53533)
0807 08:35:13.092392 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:13.092406 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:13.092616 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:13.092935 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:13.092938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:13.093217 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:13.093340 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:13.093346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:13.094220 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:13.094225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:13.094825 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:13.094834 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:13.095408 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:13.095412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:13.095501 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:13.096234 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:13.096248 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:13.098720 (+ 2472us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":124,"threads_started":1}
W20260807 08:35:13.099217 23110 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:13.092122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:53533 (local address 127.9.174.62:37703)
0807 08:35:13.092305 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:13.092310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:13.092521 (+ 211us) server_negotiation.cc:407] Connection header received
0807 08:35:13.092739 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:13.092742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:13.092805 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:13.092904 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:13.093489 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:13.094090 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:13.094963 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:13.095270 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:13.095394 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:13.098951 (+ 3557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:53533: BlockingRecv error: recv got EOF from 127.9.174.2:53533 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 08:35:13 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:13 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:13.357784 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:13.350377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:45539)
0807 08:35:13.350837 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:13.350852 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:13.351071 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:13.351418 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:13.351420 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:13.351654 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:13.351786 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:13.351793 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:13.352664 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:13.352667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:13.353316 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:13.353328 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:13.354070 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:13.354073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:13.354162 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:13.355000 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:13.355104 (+ 104us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:13.357527 (+ 2423us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":148,"threads_started":1}
W20260807 08:35:13.357803 23110 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:13.350593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:45539 (local address 127.9.174.62:37703)
0807 08:35:13.350839 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:13.350843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:13.351049 (+ 206us) server_negotiation.cc:407] Connection header received
0807 08:35:13.351176 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:13.351178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:13.351237 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:13.351318 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:13.351903 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:13.352482 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:13.353496 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:13.353804 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:13.353954 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:13.357684 (+ 3730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:45539: BlockingRecv error: recv got EOF from 127.9.174.3:45539 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 07 08:35:13 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:13 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:13.894658 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:13.887698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:32899)
0807 08:35:13.888062 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:13.888078 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:13.888201 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:13.888492 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:13.888495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:13.888743 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:13.888865 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:13.888871 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:13.889759 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:13.889762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:13.890403 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:13.890412 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:13.890962 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:13.890965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:13.891123 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:13.891961 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:13.891981 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:13.894467 (+ 2486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":123,"threads_started":1}
W20260807 08:35:13.894811 23116 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:13.887784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:32899 (local address 127.9.174.62:37703)
0807 08:35:13.888130 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:13.888134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:13.888149 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:35:13.888304 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:13.888307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:13.888360 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:13.888449 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:13.889023 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:13.889631 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:13.890545 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:13.890795 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:13.890943 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:13.894636 (+ 3693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:32899: BlockingRecv error: recv got EOF from 127.9.174.1:32899 (error 108)
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":132,"threads_started":1}
Aug 07 08:35:14 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:14 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:14.107334 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:14.100234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:50155)
0807 08:35:14.100655 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:14.100672 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:14.100795 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:14.101165 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:14.101168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:14.101424 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:14.101555 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:14.101561 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:14.102393 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:14.102396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:14.103047 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:14.103057 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:14.103624 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:14.103627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:14.103713 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:14.104459 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:14.104473 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:14.106992 (+ 2519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":166,"threads_started":1}
W20260807 08:35:14.107389 23116 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:14.100426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:50155 (local address 127.9.174.62:37703)
0807 08:35:14.100718 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:14.100722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:14.100740 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:35:14.100958 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:14.100961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:14.101020 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:14.101129 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:14.101699 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:14.102276 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:14.103209 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:14.103447 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:14.103561 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:14.107253 (+ 3692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:50155: BlockingRecv error: recv got EOF from 127.9.174.2:50155 (error 108)
Metrics: {"server-negotiator.queue_time_us":178}
Aug 07 08:35:14 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:14 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:14.370069 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:14.359084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:47319)
0807 08:35:14.359457 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:14.359472 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:14.359613 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:14.359907 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:14.359909 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:14.360155 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:14.360307 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:14.360314 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:14.361254 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:14.361257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:14.361866 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:14.361875 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:14.362453 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:14.362456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:14.362557 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:14.367264 (+ 4707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:14.367282 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:14.369854 (+ 2572us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":139,"threads_started":1}
W20260807 08:35:14.370436 23116 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:14.359153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:47319 (local address 127.9.174.62:37703)
0807 08:35:14.359322 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:14.359326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:14.359566 (+ 240us) server_negotiation.cc:407] Connection header received
0807 08:35:14.359712 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:14.359715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:14.359775 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:14.359865 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:14.360425 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:14.361139 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:14.361981 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:14.362312 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:14.362420 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:14.370099 (+ 7679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:47319: BlockingRecv error: recv got EOF from 127.9.174.3:47319 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 08:35:14 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:14 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:14.902848 23119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:14.895936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:60593)
0807 08:35:14.896324 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:14.896341 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:14.896467 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:14.896772 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:14.896775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:14.897008 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:14.897134 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:14.897140 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:14.897997 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:14.897999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:14.898608 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:14.898616 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:14.899201 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:14.899204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:14.899278 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:14.899972 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:14.899985 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:14.902602 (+ 2617us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":153,"threads_started":1}
W20260807 08:35:14.902926 23120 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:14.896164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:60593 (local address 127.9.174.62:37703)
0807 08:35:14.896442 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:14.896446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:14.896463 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:35:14.896570 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:14.896573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:14.896627 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:14.896721 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:14.897270 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:14.897885 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:14.898726 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:14.898988 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:14.899103 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:14.902771 (+ 3668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:60593: BlockingRecv error: recv got EOF from 127.9.174.1:60593 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":81,"threads_started":1}
Aug 07 08:35:15 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:15 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:15.115500 23122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:15.108529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:57241)
0807 08:35:15.108883 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:15.108901 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:15.109034 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:15.109324 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:15.109326 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:15.109575 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:15.109704 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:15.109710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:15.110714 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:15.110718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:15.111419 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:15.111430 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:15.111887 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:15.111890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:15.111970 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:15.112688 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:15.112701 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:15.115285 (+ 2584us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":129,"threads_started":1}
W20260807 08:35:15.115535 23120 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:15.108634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:57241 (local address 127.9.174.62:37703)
0807 08:35:15.108850 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:15.108854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:15.108993 (+ 139us) server_negotiation.cc:407] Connection header received
0807 08:35:15.109134 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:15.109137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:15.109194 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:15.109286 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:15.109852 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:15.110584 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:15.111535 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:15.111766 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:15.111850 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:15.115421 (+ 3571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:57241: BlockingRecv error: recv got EOF from 127.9.174.2:57241 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 07 08:35:15 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:15 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:15.378159 23123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:15.371415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:44151)
0807 08:35:15.371852 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:15.371914 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:15.372079 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:15.372402 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:15.372412 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:15.372710 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:15.372834 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:15.372841 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:15.373819 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:15.373823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:15.374434 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:15.374442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:15.374953 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:15.374957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:15.375087 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:15.375783 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:15.375796 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:15.377946 (+ 2150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":171,"threads_started":1}
W20260807 08:35:15.378319 23120 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:15.371458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:44151 (local address 127.9.174.62:37703)
0807 08:35:15.371662 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:15.371667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:15.372000 (+ 333us) server_negotiation.cc:407] Connection header received
0807 08:35:15.372201 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:15.372204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:15.372271 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:15.372375 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:15.372981 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:15.373688 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:15.374581 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:15.374819 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:15.374950 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:15.378180 (+ 3230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:44151: BlockingRecv error: recv got EOF from 127.9.174.3:44151 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 08:35:15 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:15 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:15.910808 23124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:15.904129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:37181)
0807 08:35:15.904530 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:15.904550 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:15.904684 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:15.904974 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:15.904977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:15.905215 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:15.905335 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:15.905340 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:15.906203 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:15.906206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:15.906858 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:15.906868 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:15.907416 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:15.907419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:15.907502 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:15.908084 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:15.908096 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:15.910506 (+ 2410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":134,"threads_started":1}
W20260807 08:35:15.910946 23125 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:15.904288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:37181 (local address 127.9.174.62:37703)
0807 08:35:15.904568 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:15.904572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:15.904629 (+ 57us) server_negotiation.cc:407] Connection header received
0807 08:35:15.904788 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:15.904791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:15.904843 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:15.904932 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:15.905444 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:15.906030 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:15.906976 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:15.907299 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:15.907385 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:15.910768 (+ 3383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:37181: BlockingRecv error: recv got EOF from 127.9.174.1:37181 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":92,"threads_started":1}
Aug 07 08:35:16 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:16 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:16.124131 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:16.116738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:56183)
0807 08:35:16.117177 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:16.117193 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:16.117325 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:16.117626 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:16.117629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:16.117868 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:16.118030 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:16.118037 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:16.119064 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:16.119068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:16.119747 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:16.119758 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:16.120317 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:16.120320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:16.120395 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:16.121089 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:16.121110 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:16.123958 (+ 2848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":185,"threads_started":1}
W20260807 08:35:16.124305 23125 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:16.116929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:56183 (local address 127.9.174.62:37703)
0807 08:35:16.117139 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:16.117143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:16.117345 (+ 202us) server_negotiation.cc:407] Connection header received
0807 08:35:16.117420 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:16.117423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:16.117481 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:16.117588 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:16.118181 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:16.118841 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:16.119918 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:16.120172 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:16.120283 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:16.124116 (+ 3833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:56183: BlockingRecv error: recv got EOF from 127.9.174.2:56183 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 07 08:35:16 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:16 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:16.387291 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:16.379462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:40637)
0807 08:35:16.379921 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:16.379937 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:16.380079 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:16.380536 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:16.380539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:16.380862 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:16.381006 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:16.381014 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:16.381882 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:16.381885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:16.382564 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:16.382573 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:16.383225 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:16.383230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:16.383392 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:16.384292 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:16.384306 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:16.387110 (+ 2804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":220,"threads_started":1}
W20260807 08:35:16.387581 23125 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:16.379597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:40637 (local address 127.9.174.62:37703)
0807 08:35:16.379793 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:16.379797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:16.380156 (+ 359us) server_negotiation.cc:407] Connection header received
0807 08:35:16.380218 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:16.380220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:16.380278 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:16.380515 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:16.381172 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:16.381735 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:16.382719 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:16.382959 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:16.383156 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:16.387435 (+ 4279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:40637: BlockingRecv error: recv got EOF from 127.9.174.3:40637 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 08:35:16 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:16 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:16.919560 23131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:16.911900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:52387)
0807 08:35:16.912342 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:16.912360 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:16.912507 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:16.912962 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:16.912965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:16.913230 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:16.913365 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:16.913371 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:16.914427 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:16.914431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:16.915298 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:16.915311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:16.915930 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:16.915933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:16.916012 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:16.916845 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:16.916858 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:16.919402 (+ 2544us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":140,"threads_started":1}
W20260807 08:35:16.919927 23132 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:16.911985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:52387 (local address 127.9.174.62:37703)
0807 08:35:16.912367 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:16.912372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:16.912512 (+ 140us) server_negotiation.cc:407] Connection header received
0807 08:35:16.912655 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:16.912658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:16.912736 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:16.912857 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:16.913543 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:16.914235 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:16.915466 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:16.915734 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:16.915839 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:16.919714 (+ 3875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:52387: BlockingRecv error: recv got EOF from 127.9.174.1:52387 (error 108)
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":131,"threads_started":1}
Aug 07 08:35:17 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:17 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:17.133730 23134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:17.125449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:39199)
0807 08:35:17.126033 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:17.126053 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:17.126254 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:17.126606 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:17.126609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:17.126871 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:17.126988 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:17.126995 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:17.128059 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:17.128064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:17.128767 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:17.128782 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:17.129554 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:17.129561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:17.129683 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:17.130553 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:17.130579 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:17.133534 (+ 2955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":272,"threads_started":1}
W20260807 08:35:17.134056 23132 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:17.125653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:39199 (local address 127.9.174.62:37703)
0807 08:35:17.126036 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:17.126041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:17.126168 (+ 127us) server_negotiation.cc:407] Connection header received
0807 08:35:17.126402 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:17.126405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:17.126466 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:17.126568 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:17.127181 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:17.127777 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:17.128943 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:17.129216 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:17.129353 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:17.133879 (+ 4526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:39199: BlockingRecv error: recv got EOF from 127.9.174.2:39199 (error 108)
Metrics: {"server-negotiator.queue_time_us":266}
Aug 07 08:35:17 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:17 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:17.397819 23136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:17.388799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:39413)
0807 08:35:17.389315 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:17.389335 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:17.389633 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:17.390162 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:17.390166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:17.390444 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:17.390600 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:17.390609 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:17.391757 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:17.391762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:17.392748 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:17.392768 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:17.393454 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:17.393459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:17.393578 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:17.394425 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:17.394443 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:17.397642 (+ 3199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":95,"threads_started":1}
W20260807 08:35:17.398126 23132 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:17.389014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:39413 (local address 127.9.174.62:37703)
0807 08:35:17.389242 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:17.389247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:17.389619 (+ 372us) server_negotiation.cc:407] Connection header received
0807 08:35:17.389830 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:17.389834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:17.389897 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:17.390001 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:17.390817 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:17.391504 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:17.392989 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:17.393253 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:17.393370 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:17.397944 (+ 4574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:39413: BlockingRecv error: recv got EOF from 127.9.174.3:39413 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 07 08:35:17 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:17 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:17.931452 23137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:17.920990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:43917)
0807 08:35:17.921574 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:17.921602 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:17.921784 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:17.922310 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:17.922317 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:17.922924 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:17.923346 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:17.923358 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:17.924785 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:17.924790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:17.925993 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:17.926016 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:17.926886 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:17.926891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:17.927090 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:17.928007 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:17.928036 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:17.931257 (+ 3221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":127,"threads_started":1}
W20260807 08:35:17.931777 23139 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:17.921189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:43917 (local address 127.9.174.62:37703)
0807 08:35:17.921642 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:17.921648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:17.921719 (+ 71us) server_negotiation.cc:407] Connection header received
0807 08:35:17.921927 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:17.921930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:17.922056 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:17.922222 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:17.923759 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:17.924429 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:17.926257 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:17.926625 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:17.926843 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:17.931548 (+ 4705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:43917: BlockingRecv error: recv got EOF from 127.9.174.1:43917 (error 108)
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":151,"threads_started":1}
Aug 07 08:35:18 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:18 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:18.142643 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:18.135072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:39789)
0807 08:35:18.135551 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:18.135569 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:18.135779 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:18.136183 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:18.136186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:18.136429 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:18.136566 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:18.136573 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:18.137559 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:18.137562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:18.138265 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:18.138277 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:18.138841 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:18.138844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:18.138917 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:18.139673 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:18.139692 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:18.142410 (+ 2718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":108,"threads_started":1}
W20260807 08:35:18.142753 23139 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:18.135321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:39789 (local address 127.9.174.62:37703)
0807 08:35:18.135598 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:18.135602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:18.135790 (+ 188us) server_negotiation.cc:407] Connection header received
0807 08:35:18.135874 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:18.135877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:18.135948 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:18.136068 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:18.136751 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:18.137370 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:18.138427 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:18.138695 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:18.138792 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:18.142617 (+ 3825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:39789: BlockingRecv error: recv got EOF from 127.9.174.2:39789 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
Aug 07 08:35:18 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:18 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:18.407728 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:18.399629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:56175)
0807 08:35:18.400300 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:18.400319 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:18.400459 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:18.400949 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:18.400952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:18.401218 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:18.401361 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:18.401368 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:18.402576 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:18.402581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:18.403511 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:18.403524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:18.404061 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:18.404065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:18.404157 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:18.404901 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:18.404915 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:18.407519 (+ 2604us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":233,"threads_started":1}
W20260807 08:35:18.407804 23139 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:18.399936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:56175 (local address 127.9.174.62:37703)
0807 08:35:18.400367 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:18.400375 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:18.400529 (+ 154us) server_negotiation.cc:407] Connection header received
0807 08:35:18.400605 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:18.400618 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:18.400692 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:18.400866 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:18.401567 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:18.402437 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:18.403649 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:18.403912 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:18.404051 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:18.407675 (+ 3624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:56175: BlockingRecv error: recv got EOF from 127.9.174.3:56175 (error 108)
Metrics: {"server-negotiator.queue_time_us":279}
Aug 07 08:35:18 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:18 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:18.940302 23145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:18.932755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:44885)
0807 08:35:18.933251 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:18.933272 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:18.933405 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:18.933913 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:18.933917 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:18.934179 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:18.934318 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:18.934324 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:18.935334 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:18.935337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:18.936029 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:18.936040 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:18.936642 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:18.936645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:18.936777 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:18.937586 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:18.937600 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:18.940127 (+ 2527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":185,"threads_started":1}
W20260807 08:35:18.940644 23146 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:18.933060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:44885 (local address 127.9.174.62:37703)
0807 08:35:18.933390 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:18.933395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:18.933411 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:35:18.933504 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:18.933506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:18.933564 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:18.933663 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:18.934491 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:18.935188 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:18.936183 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:18.936455 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:18.936566 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:18.940412 (+ 3846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:44885: BlockingRecv error: recv got EOF from 127.9.174.1:44885 (error 108)
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":93,"threads_started":1}
Aug 07 08:35:19 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:19 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:19.152177 23147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:19.144061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:45975)
0807 08:35:19.144444 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:19.144489 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:19.144687 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:19.145021 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:19.145025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:19.145304 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:19.145428 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:19.145434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:19.146532 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:19.146539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:19.147299 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:19.147310 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:19.147866 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:19.147870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:19.147972 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:19.148937 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:19.148952 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:19.151933 (+ 2981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":136,"threads_started":1}
W20260807 08:35:19.152387 23146 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:19.144169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:45975 (local address 127.9.174.62:37703)
0807 08:35:19.144392 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:19.144398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:19.144600 (+ 202us) server_negotiation.cc:407] Connection header received
0807 08:35:19.144812 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:19.144816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:19.144882 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:19.144997 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:19.145576 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:19.146390 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:19.147437 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:19.147705 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:19.147834 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:19.152198 (+ 4364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:45975: BlockingRecv error: recv got EOF from 127.9.174.2:45975 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 08:35:19 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:19 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:19.421762 23150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:19.409030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:50475)
0807 08:35:19.409446 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:19.409532 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:19.409677 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:19.409992 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:19.409994 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:19.410235 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:19.410359 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:19.410365 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:19.411787 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:19.411792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:19.412749 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:19.412762 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:19.413285 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:19.413288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:19.413355 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:19.418447 (+ 5092us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:19.418470 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:19.421572 (+ 3102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":200,"threads_started":1}
W20260807 08:35:19.422006 23146 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:19.409213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:50475 (local address 127.9.174.62:37703)
0807 08:35:19.409426 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:19.409431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:19.409638 (+ 207us) server_negotiation.cc:407] Connection header received
0807 08:35:19.409777 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:19.409780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:19.409841 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:19.409932 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:19.410525 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:19.411643 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:19.412909 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:19.413142 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:19.413237 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:19.421820 (+ 8583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:50475: BlockingRecv error: recv got EOF from 127.9.174.3:50475 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 08:35:19 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:19 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:19.948925 23151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:19.941432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:34281)
0807 08:35:19.941880 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:19.941897 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:19.942044 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:19.942382 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:19.942385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:19.942632 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:19.942851 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:19.942860 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:19.943840 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:19.943843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:19.944557 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:19.944568 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:19.945246 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:19.945380 (+ 134us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:19.945477 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:19.946277 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:19.946290 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:19.948704 (+ 2414us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":129,"threads_started":1}
W20260807 08:35:19.949003 23152 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:19.941628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:34281 (local address 127.9.174.62:37703)
0807 08:35:19.941923 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:19.941927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:19.942023 (+ 96us) server_negotiation.cc:407] Connection header received
0807 08:35:19.942150 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:19.942153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:19.942209 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:19.942301 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:19.943030 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:19.943700 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:19.944716 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:19.945111 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:19.945227 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:19.948868 (+ 3641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:34281: BlockingRecv error: recv got EOF from 127.9.174.1:34281 (error 108)
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":79,"threads_started":1}
Aug 07 08:35:20 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:20 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:20.160785 23154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:20.153702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:49139)
0807 08:35:20.154086 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:20.154100 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:20.154226 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:20.154595 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:20.154598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:20.154973 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:20.155151 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:20.155158 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:20.156230 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:20.156234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:20.157099 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:20.157110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:20.157589 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:20.157592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:20.157662 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:20.158277 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:20.158290 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:20.160615 (+ 2325us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":102,"threads_started":1}
W20260807 08:35:20.161159 23152 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:20.153797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:49139 (local address 127.9.174.62:37703)
0807 08:35:20.154049 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:20.154054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:20.154194 (+ 140us) server_negotiation.cc:407] Connection header received
0807 08:35:20.154363 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:20.154366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:20.154430 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:20.154532 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:20.155254 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:20.156121 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:20.157229 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:20.157476 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:20.157563 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:20.160857 (+ 3294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:49139: BlockingRecv error: recv got EOF from 127.9.174.2:49139 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 07 08:35:20 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:20 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:20.430359 23155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:20.423031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:48853)
0807 08:35:20.423436 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:20.423453 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:20.423639 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:20.423996 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:20.423999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:20.424292 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:20.424439 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:20.424447 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:20.425445 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:20.425453 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:20.426144 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:20.426154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:20.426725 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:20.426729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:20.426841 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:20.427611 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:20.427627 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:20.430184 (+ 2557us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":97,"threads_started":1}
W20260807 08:35:20.430613 23152 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:20.423143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:48853 (local address 127.9.174.62:37703)
0807 08:35:20.423347 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:20.423355 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:20.423556 (+ 201us) server_negotiation.cc:407] Connection header received
0807 08:35:20.423767 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:20.423772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:20.423845 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:20.423962 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:20.424596 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:20.425265 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:20.426297 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:20.426575 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:20.426719 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:20.430442 (+ 3723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:48853: BlockingRecv error: recv got EOF from 127.9.174.3:48853 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 08:35:20 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:20 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:20.956935 23156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:20.950101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:44735)
0807 08:35:20.950525 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:20.950543 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:20.950675 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:20.951078 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:20.951083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:20.951397 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:20.951535 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:20.951543 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:20.952411 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:20.952414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:20.953100 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:20.953111 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:20.953628 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:20.953631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:20.953704 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:20.954403 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:20.954417 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:20.956767 (+ 2350us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":135,"threads_started":1}
W20260807 08:35:20.957131 23157 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:20.950152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:44735 (local address 127.9.174.62:37703)
0807 08:35:20.950502 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:20.950507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:20.950661 (+ 154us) server_negotiation.cc:407] Connection header received
0807 08:35:20.950773 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:20.950776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:20.950829 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:20.950935 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:20.951679 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:20.952274 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:20.953239 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:20.953506 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:20.953592 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:20.956964 (+ 3372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:44735: BlockingRecv error: recv got EOF from 127.9.174.1:44735 (error 108)
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":100,"threads_started":1}
Aug 07 08:35:21 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:21 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:21.168555 23158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:21.161764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:47595)
0807 08:35:21.162179 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:21.162194 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:21.162308 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:21.162579 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:21.162582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:21.162824 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:21.162956 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:21.162963 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:21.163837 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:21.163839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:21.164459 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:21.164467 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:21.164976 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:21.164980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:21.165070 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:21.165756 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:21.165769 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:21.168323 (+ 2554us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":204,"threads_started":1}
W20260807 08:35:21.168681 23157 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:21.161845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:47595 (local address 127.9.174.62:37703)
0807 08:35:21.162050 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:21.162055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:21.162312 (+ 257us) server_negotiation.cc:407] Connection header received
0807 08:35:21.162387 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:21.162389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:21.162457 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:21.162537 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:21.163114 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:21.163732 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:21.164605 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:21.164841 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:21.164961 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:21.168547 (+ 3586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:47595: BlockingRecv error: recv got EOF from 127.9.174.2:47595 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 08:35:21 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:21 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:21.437971 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:21.431535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:34637)
0807 08:35:21.431919 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:21.431935 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:21.432086 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:21.432368 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:21.432371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:21.432616 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:21.432740 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:21.432746 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:21.433599 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:21.433601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:21.434193 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:21.434200 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:21.434703 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:21.434707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:21.434798 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:21.435664 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:21.435705 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:21.437816 (+ 2111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":137,"threads_started":1}
W20260807 08:35:21.438176 23157 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:21.431773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:34637 (local address 127.9.174.62:37703)
0807 08:35:21.431974 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:21.431978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:21.432053 (+ 75us) server_negotiation.cc:407] Connection header received
0807 08:35:21.432185 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:21.432188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:21.432246 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:21.432329 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:21.432882 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:21.433492 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:21.434330 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:21.434571 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:21.434686 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:21.437983 (+ 3297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:34637: BlockingRecv error: recv got EOF from 127.9.174.3:34637 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 07 08:35:21 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:21 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:21.965507 23165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:21.958131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:36245)
0807 08:35:21.958531 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:21.958550 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:21.958684 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:21.959071 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:21.959075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:21.959362 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:21.959539 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:21.959548 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:21.960888 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:21.960894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:21.961585 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:21.961597 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:21.962125 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:21.962128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:21.962195 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:21.962860 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:21.962874 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:21.965333 (+ 2459us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":157,"threads_started":1}
W20260807 08:35:21.965739 23167 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:21.958210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:36245 (local address 127.9.174.62:37703)
0807 08:35:21.958515 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:21.958519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:21.958640 (+ 121us) server_negotiation.cc:407] Connection header received
0807 08:35:21.958782 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:21.958785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:21.958862 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:21.958987 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:21.959734 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:21.960685 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:21.961729 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:21.961995 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:21.962095 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:21.965526 (+ 3431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:36245: BlockingRecv error: recv got EOF from 127.9.174.1:36245 (error 108)
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":112,"threads_started":1}
Aug 07 08:35:22 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:22 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:22.176553 23168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:22.169757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:38021)
0807 08:35:22.170181 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:22.170200 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:22.170345 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:22.170666 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:22.170669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:22.170881 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:22.171106 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:22.171114 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:22.172006 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:22.172009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:22.172601 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:22.172609 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:22.173118 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:22.173120 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:22.173191 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:22.173798 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:22.173811 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:22.176362 (+ 2551us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":117,"threads_started":1}
W20260807 08:35:22.176731 23167 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:22.169811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:38021 (local address 127.9.174.62:37703)
0807 08:35:22.170002 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:22.170007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:22.170275 (+ 268us) server_negotiation.cc:407] Connection header received
0807 08:35:22.170451 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:22.170455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:22.170523 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:22.170637 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:22.171219 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:22.171863 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:22.172728 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:22.172997 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:22.173090 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:22.176549 (+ 3459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:38021: BlockingRecv error: recv got EOF from 127.9.174.2:38021 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 08:35:22 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:22 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:22.445384 23171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:22.438937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:53989)
0807 08:35:22.439291 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:22.439305 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:22.439427 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:22.439728 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:22.439731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:22.440003 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:22.440126 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:22.440132 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:22.441024 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:22.441027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:22.441644 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:22.441652 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:22.442188 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:22.442191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:22.442276 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:22.442888 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:22.442901 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:22.445202 (+ 2301us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":146,"threads_started":1}
W20260807 08:35:22.445554 23167 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:22.439084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:53989 (local address 127.9.174.62:37703)
0807 08:35:22.439256 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:22.439260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:22.439384 (+ 124us) server_negotiation.cc:407] Connection header received
0807 08:35:22.439517 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:22.439520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:22.439580 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:22.439656 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:22.440276 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:22.440880 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:22.441774 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:22.442049 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:22.442148 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:22.445402 (+ 3254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:53989: BlockingRecv error: recv got EOF from 127.9.174.3:53989 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 08:35:22 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:22 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:22.973379 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:22.966600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:49403)
0807 08:35:22.966949 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:22.966968 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:22.967145 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:22.967489 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:22.967492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:22.967722 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:22.967844 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:22.967849 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:22.968823 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:22.968825 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:22.969422 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:22.969430 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:22.969996 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:22.969999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:22.970111 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:22.970682 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:22.970695 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:22.973173 (+ 2478us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":116,"threads_started":1}
W20260807 08:35:22.973567 23177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:22.966708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:49403 (local address 127.9.174.62:37703)
0807 08:35:22.967120 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:22.967124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:22.967148 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:35:22.967245 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:22.967248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:22.967316 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:22.967456 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:22.967990 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:22.968702 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:22.969648 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:22.969888 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:22.969979 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:22.973394 (+ 3415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:49403: BlockingRecv error: recv got EOF from 127.9.174.1:49403 (error 108)
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":186,"threads_started":1}
Aug 07 08:35:23 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:23 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:23.184291 23178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:23.177565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:41107)
0807 08:35:23.178077 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:23.178091 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:23.178252 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:23.178597 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:23.178600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:23.178819 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:23.178945 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:23.178958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:23.180051 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:23.180054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:23.180681 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:23.180689 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:23.181261 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:23.181263 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:23.181331 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:23.181973 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:23.181988 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:23.184139 (+ 2151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":135,"threads_started":1}
W20260807 08:35:23.184572 23177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:23.177687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:41107 (local address 127.9.174.62:37703)
0807 08:35:23.177881 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:23.177888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:23.178180 (+ 292us) server_negotiation.cc:407] Connection header received
0807 08:35:23.178391 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:23.178394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:23.178470 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:23.178558 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:23.179158 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:23.179914 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:23.180825 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:23.181132 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:23.181241 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:23.184410 (+ 3169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:41107: BlockingRecv error: recv got EOF from 127.9.174.2:41107 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 08:35:23 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:23 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:23.453464 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:23.446474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:45481)
0807 08:35:23.446939 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:23.446955 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:23.447172 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:23.447523 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:23.447526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:23.447800 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:23.447939 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:23.447946 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:23.448932 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:23.448936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:23.449535 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:23.449543 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:23.450048 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:23.450051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:23.450147 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:23.450856 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:23.450869 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:23.453240 (+ 2371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":125,"threads_started":1}
W20260807 08:35:23.453533 23177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:23.446652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:45481 (local address 127.9.174.62:37703)
0807 08:35:23.446837 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:23.446842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:23.447070 (+ 228us) server_negotiation.cc:407] Connection header received
0807 08:35:23.447288 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:23.447294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:23.447362 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:23.447494 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:23.448121 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:23.448794 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:23.449664 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:23.449911 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:23.450037 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:23.453386 (+ 3349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:45481: BlockingRecv error: recv got EOF from 127.9.174.3:45481 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 08:35:23 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:23 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:23.981951 23180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:23.974334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:58415)
0807 08:35:23.974724 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:23.974739 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:23.974867 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:23.975301 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:23.975305 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:23.975595 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:23.975732 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:23.975740 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:23.976725 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:23.976729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:23.977475 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:23.977488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:23.978106 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:23.978110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:23.978200 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:23.979114 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:23.979130 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:23.981715 (+ 2585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":144,"threads_started":1}
W20260807 08:35:23.982185 23182 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:23.974390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:58415 (local address 127.9.174.62:37703)
0807 08:35:23.974724 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:23.974728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:23.974823 (+ 95us) server_negotiation.cc:407] Connection header received
0807 08:35:23.975066 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:23.975069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:23.975125 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:23.975214 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:23.975906 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:23.976581 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:23.977638 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:23.977945 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:23.978057 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:23.981884 (+ 3827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:58415: BlockingRecv error: recv got EOF from 127.9.174.1:58415 (error 108)
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":112,"threads_started":1}
Aug 07 08:35:24 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:24 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:24.191776 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:24.185401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:47019)
0807 08:35:24.185760 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:24.185774 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:24.185899 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:24.186168 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:24.186171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:24.186404 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:24.186532 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:24.186538 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:24.187417 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:24.187419 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:24.188008 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:24.188016 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:24.188452 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:24.188454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:24.188526 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:24.189263 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:24.189276 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:24.191596 (+ 2320us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":138,"threads_started":1}
W20260807 08:35:24.191984 23182 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:24.185538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:47019 (local address 127.9.174.62:37703)
0807 08:35:24.185728 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:24.185732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:24.185854 (+ 122us) server_negotiation.cc:407] Connection header received
0807 08:35:24.185981 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:24.185983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:24.186054 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:24.186133 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:24.186644 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:24.187316 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:24.188118 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:24.188343 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:24.188428 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:24.191788 (+ 3360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:47019: BlockingRecv error: recv got EOF from 127.9.174.2:47019 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 08:35:24 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:24 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:24.471231 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:24.454689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:47437)
0807 08:35:24.455291 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:24.455306 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:24.455449 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:24.455875 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:24.455878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:24.456131 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:24.456307 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:24.456315 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:24.457429 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:24.457435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:24.458238 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:24.458251 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:24.458793 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:24.458797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:24.458907 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:24.468182 (+ 9275us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:24.468206 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:24.470941 (+ 2735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":189,"threads_started":1}
W20260807 08:35:24.471460 23182 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:24.454875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:47437 (local address 127.9.174.62:37703)
0807 08:35:24.455175 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:24.455179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:24.455401 (+ 222us) server_negotiation.cc:407] Connection header received
0807 08:35:24.455574 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:24.455577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:24.455672 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:24.455846 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:24.456453 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:24.457290 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:24.458397 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:24.458660 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:24.458783 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:24.471314 (+ 12531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:47437: BlockingRecv error: recv got EOF from 127.9.174.3:47437 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 07 08:35:24 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:24 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:24.990521 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:24.982970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:35495)
0807 08:35:24.983390 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:24.983406 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:24.983537 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:24.983854 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:24.983857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:24.984136 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:24.984262 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:24.984268 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:24.985237 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:24.985240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:24.985921 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:24.985933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:24.986560 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:24.986563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:24.986665 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:24.987546 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:24.987561 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:24.990360 (+ 2799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":128,"threads_started":1}
W20260807 08:35:24.990814 23192 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:24.983166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:35495 (local address 127.9.174.62:37703)
0807 08:35:24.983534 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:24.983538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:24.983556 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:35:24.983636 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:24.983638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:24.983692 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:24.983780 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:24.984398 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:24.985074 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:24.986088 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:24.986403 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:24.986548 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:24.990546 (+ 3998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:35495: BlockingRecv error: recv got EOF from 127.9.174.1:35495 (error 108)
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":106,"threads_started":1}
Aug 07 08:35:25 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:25 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:25.199996 23194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:25.192900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:59053)
0807 08:35:25.193318 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:25.193342 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:25.193529 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:25.193837 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:25.193841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:25.194123 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:25.194257 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:25.194264 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:25.195154 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:25.195159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:25.195777 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:25.195786 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:25.196273 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:25.196276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:25.196363 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:25.197037 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:25.197050 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:25.199707 (+ 2657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":125,"threads_started":1}
W20260807 08:35:25.200057 23192 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:25.193034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:59053 (local address 127.9.174.62:37703)
0807 08:35:25.193284 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:25.193289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:25.193445 (+ 156us) server_negotiation.cc:407] Connection header received
0807 08:35:25.193645 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:25.193649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:25.193711 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:25.193814 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:25.194383 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:25.194978 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:25.195913 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:25.196144 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:25.196260 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:25.199913 (+ 3653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:59053: BlockingRecv error: recv got EOF from 127.9.174.2:59053 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Aug 07 08:35:25 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:25 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.3@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:25.479353 23198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:25.472367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.3:43187)
0807 08:35:25.472837 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:25.472860 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:25.473049 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:25.473516 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:25.473520 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:25.473868 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:25.473999 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:25.474007 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:25.475028 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:25.475031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:25.475706 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:25.475716 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:25.476322 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:25.476325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:25.476409 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:25.477136 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:25.477149 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:25.479193 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":116,"threads_started":1}
W20260807 08:35:25.479809 23192 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:25.472520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:43187 (local address 127.9.174.62:37703)
0807 08:35:25.472692 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:25.472698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:25.472979 (+ 281us) server_negotiation.cc:407] Connection header received
0807 08:35:25.473252 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:25.473256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:25.473337 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:25.473467 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:25.474211 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:25.474865 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:25.475865 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:25.476148 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:25.476283 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:25.479607 (+ 3324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.3:43187: BlockingRecv error: recv got EOF from 127.9.174.3:43187 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 08:35:25 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:25 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.1@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:25.998683 23199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:25.991940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.1:47091)
0807 08:35:25.992392 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:25.992411 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:25.992520 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:25.992944 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:25.992952 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:25.993228 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:25.993405 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:25.993412 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:25.994576 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:25.994579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:25.995262 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:25.995273 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:25.995831 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:25.995834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:25.995901 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:25.996557 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:25.996570 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:25.998538 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":130,"threads_started":1}
W20260807 08:35:25.999169 23201 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:25.992096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:47091 (local address 127.9.174.62:37703)
0807 08:35:25.992434 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:25.992439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:25.992462 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:35:25.992706 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:25.992709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:25.992773 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:25.992877 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:25.993579 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:25.994428 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:25.995409 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:25.995686 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:25.995793 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:25.998840 (+ 3047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.1:47091: BlockingRecv error: recv got EOF from 127.9.174.1:47091 (error 108)
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":81,"threads_started":1}
Aug 07 08:35:26 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:35:26 dist-test-slave-h3ft krb5kdc[21582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.174.2@KRBTEST.COM for oryx/127.9.174.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:35:26.208456 23202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:26.201547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37703 (local address 127.9.174.2:54055)
0807 08:35:26.201920 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:26.201936 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:26.202064 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:26.202402 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:26.202405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:26.202662 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:26.202793 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:26.202798 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:26.203754 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:26.203758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:26.204494 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:26.204505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:26.205074 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:26.205078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:26.205201 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:26.205893 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:26.205906 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:26.208229 (+ 2323us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.174.62:37703: Server oryx/127.9.174.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":131,"threads_started":1}
W20260807 08:35:26.208550 23201 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:26.201687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:54055 (local address 127.9.174.62:37703)
0807 08:35:26.201945 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:26.201949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:26.202033 (+ 84us) server_negotiation.cc:407] Connection header received
0807 08:35:26.202180 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:26.202183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:26.202245 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:26.202355 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:26.202948 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:26.203589 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:26.204662 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:26.204930 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:26.205066 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:26.208413 (+ 3347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.174.2:54055: BlockingRecv error: recv got EOF from 127.9.174.2:54055 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 08:35:26.310694 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 22446
I20260807 08:35:26.335122 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 22689
I20260807 08:35:26.354750 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 22889
I20260807 08:35:26.370570 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 22341
2026-08-07T08:35:26Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18760 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T08:35:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:35:26Z Disabled control of system clock
I20260807 08:35:26.420043 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:39189
--webserver_interface=127.9.174.62
--webserver_port=0
--builtin_ntp_servers=127.9.174.20:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:39189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 08:35:26.562744 23208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:26.563197 23208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:26.563278 23208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:26.567584 23208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:35:26.567663 23208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:26.567682 23208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:26.567750 23208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:35:26.567786 23208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:35:26.573189 23208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:39189
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:39189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:26.574855 23208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:26.576892 23208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:26.588842 23214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:26.589025 23208 server_base.cc:1034] running on GCE node
W20260807 08:35:26.588845 23215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:26.589283 23218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:26.590196 23208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:26.591887 23208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:26.593146 23208 hybrid_clock.cc:648] HybridClock initialized: now 1786091726593113 us; error 55 us; skew 500 ppm
I20260807 08:35:26.595855 23208 webserver.cc:460] Webserver started at http://127.9.174.62:44335/ using document root <none> and password file <none>
I20260807 08:35:26.596692 23208 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:26.596807 23208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:26.597105 23208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:26.599895 23208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "da2d68e078a0418c89ae236650e465f6"
format_stamp: "Formatted at 2026-08-07 08:35:26 on dist-test-slave-h3ft"
server_key: "7ab7021872f67ea9e98cb04255bf290e"
server_key_iv: "00a085cea676770b4164eef1e9baa6c0"
server_key_version: "encryptionkey@0"
I20260807 08:35:26.600592 23208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "da2d68e078a0418c89ae236650e465f6"
format_stamp: "Formatted at 2026-08-07 08:35:26 on dist-test-slave-h3ft"
server_key: "7ab7021872f67ea9e98cb04255bf290e"
server_key_iv: "00a085cea676770b4164eef1e9baa6c0"
server_key_version: "encryptionkey@0"
I20260807 08:35:26.605579 23208 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:35:26.608767 23226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:26.610471 23208 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:35:26.610707 23208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "da2d68e078a0418c89ae236650e465f6"
format_stamp: "Formatted at 2026-08-07 08:35:26 on dist-test-slave-h3ft"
server_key: "7ab7021872f67ea9e98cb04255bf290e"
server_key_iv: "00a085cea676770b4164eef1e9baa6c0"
server_key_version: "encryptionkey@0"
I20260807 08:35:26.610886 23208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:26.628707 23208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:26.629720 23208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:26.630002 23208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:26.639444 23208 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:39189
I20260807 08:35:26.639473 23312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:39189 every 8 connection(s)
I20260807 08:35:26.640664 23208 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:35:26.644811 23313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:26.650242 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 23208
I20260807 08:35:26.650393 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:35:26.650784 9912 external_mini_cluster.cc:1428] Setting key 509d283258dc5483c3a69a687f950324
I20260807 08:35:26.652318 23313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6: Bootstrap starting.
I20260807 08:35:26.655257 23313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:26.656306 23313 log.cc:824] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6: Log is configured to *not* fsync() on all Append() calls
I20260807 08:35:26.658581 23313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6: No bootstrap required, opened a new log
I20260807 08:35:26.662366 23317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:26.652795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60728 (local address 127.9.174.62:39189)
0807 08:35:26.653633 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:26.653655 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:26.653700 (+ 45us) server_negotiation.cc:407] Connection header received
0807 08:35:26.654479 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:26.654499 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:26.654992 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:26.655472 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:26.655949 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:26.657095 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:26.658158 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:26.658755 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:26.658955 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:26.660127 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:26.660133 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:26.660150 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:35:26.660648 (+ 498us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:26.660725 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:35:26.660750 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:26.660880 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:26.661038 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 08:35:26.661391 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":178,"threads_started":1}
I20260807 08:35:26.663477 23313 raft_consensus.cc:348] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2d68e078a0418c89ae236650e465f6" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39189 } }
I20260807 08:35:26.663808 23313 raft_consensus.cc:374] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:26.663862 23313 raft_consensus.cc:729] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da2d68e078a0418c89ae236650e465f6, State: Initialized, Role: FOLLOWER
I20260807 08:35:26.664422 23313 consensus_queue.cc:260] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2d68e078a0418c89ae236650e465f6" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39189 } }
I20260807 08:35:26.664604 23313 raft_consensus.cc:388] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:35:26.664670 23313 raft_consensus.cc:482] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:35:26.664780 23313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:26.666152 23313 raft_consensus.cc:504] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2d68e078a0418c89ae236650e465f6" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39189 } }
I20260807 08:35:26.666518 23313 leader_election.cc:302] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da2d68e078a0418c89ae236650e465f6; no voters:
I20260807 08:35:26.666846 23313 leader_election.cc:288] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:35:26.667042 23319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:35:26.667369 23319 raft_consensus.cc:686] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [term 1 LEADER]: Becoming Leader. State: Replica: da2d68e078a0418c89ae236650e465f6, State: Running, Role: LEADER
I20260807 08:35:26.667781 23319 consensus_queue.cc:237] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2d68e078a0418c89ae236650e465f6" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39189 } }
I20260807 08:35:26.668471 23313 sys_catalog.cc:564] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:35:26.669814 23320 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da2d68e078a0418c89ae236650e465f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2d68e078a0418c89ae236650e465f6" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39189 } } }
I20260807 08:35:26.669965 23320 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [sys.catalog]: This master's current role is: LEADER
I20260807 08:35:26.670423 23335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:35:26.670382 23321 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da2d68e078a0418c89ae236650e465f6. Latest consensus state: current_term: 1 leader_uuid: "da2d68e078a0418c89ae236650e465f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2d68e078a0418c89ae236650e465f6" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39189 } } }
I20260807 08:35:26.670603 23321 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6 [sys.catalog]: This master's current role is: LEADER
I20260807 08:35:26.673911 23335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:35:26.681486 23335 catalog_manager.cc:1347] Generated new cluster ID: 7b42a5a8fc9c403c9257f30c278fa7f9
I20260807 08:35:26.681610 23335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:35:26.708756 23335 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:35:26.710093 23335 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:35:26.725972 23335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da2d68e078a0418c89ae236650e465f6: Generated new TSK 0
I20260807 08:35:26.727088 23335 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:35:26.730566 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:39189
--builtin_ntp_servers=127.9.174.20:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 08:35:26.878572 23354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:26.878943 23354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:26.878990 23354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:26.883486 23354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:26.883586 23354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:26.883678 23354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:35:26.888994 23354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:39189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:26.890668 23354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:26.892681 23354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:26.904861 23363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:26.904861 23361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:26.904861 23367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:26.905227 23354 server_base.cc:1034] running on GCE node
I20260807 08:35:26.905906 23354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:26.906787 23354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:26.908001 23354 hybrid_clock.cc:648] HybridClock initialized: now 1786091726907952 us; error 64 us; skew 500 ppm
I20260807 08:35:26.910519 23354 webserver.cc:460] Webserver started at http://127.9.174.1:38945/ using document root <none> and password file <none>
I20260807 08:35:26.911329 23354 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:26.911433 23354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:26.911726 23354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:26.914373 23354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "4986f51bb57640e1abce8fc8858fde77"
format_stamp: "Formatted at 2026-08-07 08:35:26 on dist-test-slave-h3ft"
server_key: "c90f88d52244a91fa86050c1212a0294"
server_key_iv: "52b2c98eabcfb107f0f5cdf597b584c3"
server_key_version: "encryptionkey@0"
I20260807 08:35:26.915194 23354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "4986f51bb57640e1abce8fc8858fde77"
format_stamp: "Formatted at 2026-08-07 08:35:26 on dist-test-slave-h3ft"
server_key: "c90f88d52244a91fa86050c1212a0294"
server_key_iv: "52b2c98eabcfb107f0f5cdf597b584c3"
server_key_version: "encryptionkey@0"
I20260807 08:35:26.920557 23354 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 08:35:26.924068 23376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:26.925720 23354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:35:26.925886 23354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "4986f51bb57640e1abce8fc8858fde77"
format_stamp: "Formatted at 2026-08-07 08:35:26 on dist-test-slave-h3ft"
server_key: "c90f88d52244a91fa86050c1212a0294"
server_key_iv: "52b2c98eabcfb107f0f5cdf597b584c3"
server_key_version: "encryptionkey@0"
I20260807 08:35:26.926075 23354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:26.946761 23354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:26.947980 23354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:26.948312 23354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:26.949199 23354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:35:26.950554 23354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:26.950649 23354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:26.950742 23354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:26.950804 23354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:26.963092 23354 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:44479
I20260807 08:35:26.963151 23552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:44479 every 8 connection(s)
I20260807 08:35:26.964418 23354 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:35:26.971498 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 23354
I20260807 08:35:26.971644 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:35:26.972056 9912 external_mini_cluster.cc:1428] Setting key e325a2ff086e8335824a7aeb0b0028be
I20260807 08:35:26.974857 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:39189
--builtin_ntp_servers=127.9.174.20:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 08:35:26.978446 23317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:26.967181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:36225 (local address 127.9.174.62:39189)
0807 08:35:26.967389 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:26.967395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:26.968902 (+ 1507us) server_negotiation.cc:407] Connection header received
0807 08:35:26.971072 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:26.971077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:26.971171 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:26.971296 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:26.972806 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:26.973565 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:26.975372 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:26.975892 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:26.976042 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:26.977639 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:26.977645 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:26.977651 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:35:26.977672 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:26.977708 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:35:26.977724 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:26.977829 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:26.978091 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 08:35:26.978190 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 08:35:26.979770 23560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:26.967706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:39189 (local address 127.9.174.1:36225)
0807 08:35:26.968648 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:26.968705 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:26.970646 (+ 1941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:26.971524 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:26.971533 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:26.971696 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:26.972555 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:26.972593 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:26.973749 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:26.973757 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:26.975220 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:26.975233 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:26.976119 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:26.976122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:26.976328 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:26.977270 (+ 942us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:35:26.977271 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:26.977339 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:35:26.977442 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:26.977878 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:26.977889 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:26.977896 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 08:35:26.978068 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 08:35:26.978396 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":178,"threads_started":1}
I20260807 08:35:26.981544 23553 heartbeater.cc:347] Connected to a master server at 127.9.174.62:39189
I20260807 08:35:26.982028 23553 heartbeater.cc:464] Registering TS with master...
I20260807 08:35:26.982813 23553 heartbeater.cc:511] Master 127.9.174.62:39189 requested a full tablet report, sending...
I20260807 08:35:26.985083 23259 ts_manager.cc:194] Registered new tserver with Master: 4986f51bb57640e1abce8fc8858fde77 (127.9.174.1:44479)
I20260807 08:35:26.986596 23259 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.174.1:36225
W20260807 08:35:27.112479 23561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:27.112864 23561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:27.112911 23561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:27.117581 23561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:27.117681 23561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:27.117748 23561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:35:27.123194 23561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:39189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:27.124751 23561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:27.126622 23561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:27.139536 23568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:27.139494 23570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:27.139499 23567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:27.140136 23561 server_base.cc:1034] running on GCE node
I20260807 08:35:27.140729 23561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:27.141703 23561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:27.142980 23561 hybrid_clock.cc:648] HybridClock initialized: now 1786091727142948 us; error 48 us; skew 500 ppm
I20260807 08:35:27.145628 23561 webserver.cc:460] Webserver started at http://127.9.174.2:34055/ using document root <none> and password file <none>
I20260807 08:35:27.146397 23561 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:27.146505 23561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:27.146801 23561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:27.149559 23561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "bef2a2d0aa75435da31e8daad54a9a96"
format_stamp: "Formatted at 2026-08-07 08:35:27 on dist-test-slave-h3ft"
server_key: "940b9d345e5496612f7d4b890eb6bcc2"
server_key_iv: "1a1161da69cbabcb89836772e0d6c851"
server_key_version: "encryptionkey@0"
I20260807 08:35:27.150262 23561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "bef2a2d0aa75435da31e8daad54a9a96"
format_stamp: "Formatted at 2026-08-07 08:35:27 on dist-test-slave-h3ft"
server_key: "940b9d345e5496612f7d4b890eb6bcc2"
server_key_iv: "1a1161da69cbabcb89836772e0d6c851"
server_key_version: "encryptionkey@0"
I20260807 08:35:27.154826 23561 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 08:35:27.158260 23578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:27.159838 23561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:35:27.159973 23561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "bef2a2d0aa75435da31e8daad54a9a96"
format_stamp: "Formatted at 2026-08-07 08:35:27 on dist-test-slave-h3ft"
server_key: "940b9d345e5496612f7d4b890eb6bcc2"
server_key_iv: "1a1161da69cbabcb89836772e0d6c851"
server_key_version: "encryptionkey@0"
I20260807 08:35:27.160146 23561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:27.192795 23561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:27.193810 23561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:27.194056 23561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:27.194841 23561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:35:27.196205 23561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:27.196256 23561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:27.196334 23561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:27.196373 23561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:27.207209 23561 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:36053
I20260807 08:35:27.207247 23757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:36053 every 8 connection(s)
I20260807 08:35:27.208379 23561 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:35:27.216274 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 23561
I20260807 08:35:27.216430 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:35:27.216784 9912 external_mini_cluster.cc:1428] Setting key be21b71e747ebc4b055761a3249c96e8
I20260807 08:35:27.219425 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:39189
--builtin_ntp_servers=127.9.174.20:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 08:35:27.221035 23317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:27.210796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:51085 (local address 127.9.174.62:39189)
0807 08:35:27.211052 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:27.211058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:27.212494 (+ 1436us) server_negotiation.cc:407] Connection header received
0807 08:35:27.214334 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:27.214338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:27.214412 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:27.214503 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:27.215732 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:27.216647 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:27.218506 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:27.218769 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:27.218898 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:27.220409 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:27.220414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:27.220417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:35:27.220439 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:27.220463 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:35:27.220473 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:27.220573 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:27.220767 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 08:35:27.220852 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 08:35:27.222249 23765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:27.211377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:39189 (local address 127.9.174.2:51085)
0807 08:35:27.212296 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:27.212356 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:27.213999 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:27.214709 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:27.214717 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:27.214814 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:27.215569 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:27.215587 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:27.216801 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:27.216808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:27.218341 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:27.218357 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:27.218953 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:27.218957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:27.219196 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:27.220067 (+ 871us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:35:27.220069 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:27.220129 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:35:27.220222 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:27.220578 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:27.220592 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:27.220606 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 08:35:27.220756 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 08:35:27.221171 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":159,"threads_started":1}
I20260807 08:35:27.223927 23761 heartbeater.cc:347] Connected to a master server at 127.9.174.62:39189
I20260807 08:35:27.224293 23761 heartbeater.cc:464] Registering TS with master...
I20260807 08:35:27.225240 23761 heartbeater.cc:511] Master 127.9.174.62:39189 requested a full tablet report, sending...
I20260807 08:35:27.227123 23259 ts_manager.cc:194] Registered new tserver with Master: bef2a2d0aa75435da31e8daad54a9a96 (127.9.174.2:36053)
I20260807 08:35:27.227874 23259 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.174.2:51085
W20260807 08:35:27.371752 23769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:27.372170 23769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:27.372219 23769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:27.376945 23769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:27.377051 23769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:27.377120 23769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:35:27.382962 23769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:39189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:27.384819 23769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:27.386860 23769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:27.400020 23781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:27.400077 23769 server_base.cc:1034] running on GCE node
W20260807 08:35:27.400036 23779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:27.400020 23778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:27.400955 23769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:27.401780 23769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:27.403069 23769 hybrid_clock.cc:648] HybridClock initialized: now 1786091727402986 us; error 39 us; skew 500 ppm
I20260807 08:35:27.405524 23769 webserver.cc:460] Webserver started at http://127.9.174.3:44117/ using document root <none> and password file <none>
I20260807 08:35:27.406337 23769 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:27.406422 23769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:27.406702 23769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:27.409493 23769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "250b9866edae4a41a90aa813cbcd532d"
format_stamp: "Formatted at 2026-08-07 08:35:27 on dist-test-slave-h3ft"
server_key: "9e6e9ee36d9cd8f1a0ddfbb60ff842ae"
server_key_iv: "75f728642ec45dc765be39787b713656"
server_key_version: "encryptionkey@0"
I20260807 08:35:27.410218 23769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "250b9866edae4a41a90aa813cbcd532d"
format_stamp: "Formatted at 2026-08-07 08:35:27 on dist-test-slave-h3ft"
server_key: "9e6e9ee36d9cd8f1a0ddfbb60ff842ae"
server_key_iv: "75f728642ec45dc765be39787b713656"
server_key_version: "encryptionkey@0"
I20260807 08:35:27.415458 23769 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 08:35:27.419207 23792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:27.420990 23769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:35:27.421144 23769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "250b9866edae4a41a90aa813cbcd532d"
format_stamp: "Formatted at 2026-08-07 08:35:27 on dist-test-slave-h3ft"
server_key: "9e6e9ee36d9cd8f1a0ddfbb60ff842ae"
server_key_iv: "75f728642ec45dc765be39787b713656"
server_key_version: "encryptionkey@0"
I20260807 08:35:27.421283 23769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:27.433527 23769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:27.434420 23769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:27.434669 23769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:27.435559 23769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:35:27.436789 23769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:27.436872 23769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:27.436956 23769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:27.437005 23769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:27.449199 23769 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:32973
I20260807 08:35:27.449406 23974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:32973 every 8 connection(s)
I20260807 08:35:27.450368 23769 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:35:27.451829 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 23769
I20260807 08:35:27.452008 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:35:27.452437 9912 external_mini_cluster.cc:1428] Setting key b444b4c947b6f2db8af7d19c25d26884
I20260807 08:35:27.463467 23317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:27.453460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:58607 (local address 127.9.174.62:39189)
0807 08:35:27.453683 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:27.453689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:27.455378 (+ 1689us) server_negotiation.cc:407] Connection header received
0807 08:35:27.457289 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:27.457292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:27.457372 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:27.457476 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:27.458746 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:27.459544 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:27.460966 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:27.461203 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:27.461318 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:27.462623 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:27.462627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:27.462629 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:35:27.462653 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:27.462679 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:35:27.462687 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:27.462755 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:27.463167 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 08:35:27.463313 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 08:35:27.464752 23978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:27.453995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:39189 (local address 127.9.174.3:58607)
0807 08:35:27.455097 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:27.455176 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:27.457003 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:27.457699 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:27.457707 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:27.457812 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:27.458563 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:27.458579 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:27.459703 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:27.459712 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:27.460821 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:27.460833 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:27.461374 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:27.461377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:27.461552 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:27.462317 (+ 765us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:35:27.462319 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:27.462369 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:35:27.462453 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:27.462814 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:27.462825 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:27.462831 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:35:27.463154 (+ 323us) client_negotiation.cc:240] Negotiation successful
0807 08:35:27.463533 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":142,"threads_started":1}
I20260807 08:35:27.466516 23976 heartbeater.cc:347] Connected to a master server at 127.9.174.62:39189
I20260807 08:35:27.466974 23976 heartbeater.cc:464] Registering TS with master...
I20260807 08:35:27.467891 23976 heartbeater.cc:511] Master 127.9.174.62:39189 requested a full tablet report, sending...
I20260807 08:35:27.469379 23259 ts_manager.cc:194] Registered new tserver with Master: 250b9866edae4a41a90aa813cbcd532d (127.9.174.3:32973)
I20260807 08:35:27.470144 23259 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.174.3:58607
I20260807 08:35:27.481010 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 08:35:27.485150 23317 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:27.483751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.9.174.62:39189)
0807 08:35:27.483997 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:27.484004 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:27.484242 (+ 238us) server_negotiation.cc:407] Connection header received
0807 08:35:27.484449 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:27.484452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:27.484505 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:27.484600 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:27.484943 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60736: BlockingRecv error: recv got EOF from 127.0.0.1:60736 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 08:35:27.485842 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 23354
I20260807 08:35:27.500662 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 23561
I20260807 08:35:27.523113 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 23769
I20260807 08:35:27.535492 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 23208
2026-08-07T08:35:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1163 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T08:35:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:35:27Z Disabled control of system clock
I20260807 08:35:27.585239 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:42281
--webserver_interface=127.9.174.62
--webserver_port=0
--builtin_ntp_servers=127.9.174.20:43137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:42281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 08:35:27.719554 23989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:27.719899 23989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:27.719944 23989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:27.724655 23989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:35:27.724754 23989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:27.724771 23989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:27.724792 23989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:35:27.724808 23989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:35:27.730791 23989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:42281
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:42281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:27.732244 23989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:27.733927 23989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:27.744591 23999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:27.744580 23989 server_base.cc:1034] running on GCE node
W20260807 08:35:27.744570 24000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:27.744638 24002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:27.745559 23989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:27.747141 23989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:27.748374 23989 hybrid_clock.cc:648] HybridClock initialized: now 1786091727748343 us; error 47 us; skew 500 ppm
I20260807 08:35:27.754923 23989 webserver.cc:460] Webserver started at http://127.9.174.62:45793/ using document root <none> and password file <none>
I20260807 08:35:27.755877 23989 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:27.755973 23989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:27.756271 23989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:27.759153 23989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "075863553e9348e4b914efc0073f7382"
format_stamp: "Formatted at 2026-08-07 08:35:27 on dist-test-slave-h3ft"
server_key: "934fe9931391912914c29a05f6a76bd1"
server_key_iv: "3bd551d3d4529d81de4dc436f25cec6c"
server_key_version: "encryptionkey@0"
I20260807 08:35:27.759888 23989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "075863553e9348e4b914efc0073f7382"
format_stamp: "Formatted at 2026-08-07 08:35:27 on dist-test-slave-h3ft"
server_key: "934fe9931391912914c29a05f6a76bd1"
server_key_iv: "3bd551d3d4529d81de4dc436f25cec6c"
server_key_version: "encryptionkey@0"
I20260807 08:35:27.765021 23989 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 08:35:27.768512 24010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:27.770480 23989 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:35:27.770678 23989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "075863553e9348e4b914efc0073f7382"
format_stamp: "Formatted at 2026-08-07 08:35:27 on dist-test-slave-h3ft"
server_key: "934fe9931391912914c29a05f6a76bd1"
server_key_iv: "3bd551d3d4529d81de4dc436f25cec6c"
server_key_version: "encryptionkey@0"
I20260807 08:35:27.770851 23989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:27.777144 23989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:27.778033 23989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:27.778271 23989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:27.787992 24087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:42281 every 8 connection(s)
I20260807 08:35:27.787992 23989 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:42281
I20260807 08:35:27.789517 23989 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:35:27.793691 24089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:27.794638 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 23989
I20260807 08:35:27.794783 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:35:27.795184 9912 external_mini_cluster.cc:1428] Setting key b965c3b939bbbb033ee8b02fdc8d41fb
I20260807 08:35:27.802744 24089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382: Bootstrap starting.
I20260807 08:35:27.803683 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:27.797326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42704 (local address 127.9.174.62:42281)
0807 08:35:27.798347 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:27.798371 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:27.798430 (+ 59us) server_negotiation.cc:407] Connection header received
0807 08:35:27.799494 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:27.799523 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:27.800071 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:27.800505 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:27.801646 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:27.801652 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:27.801668 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:35:27.801680 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:27.801757 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:35:27.801777 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:27.801997 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:27.802072 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 08:35:27.802489 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":246,"threads_started":1}
I20260807 08:35:27.805392 24089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:27.806229 24089 log.cc:824] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382: Log is configured to *not* fsync() on all Append() calls
I20260807 08:35:27.808925 24089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382: No bootstrap required, opened a new log
I20260807 08:35:27.813061 24089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075863553e9348e4b914efc0073f7382" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42281 } }
I20260807 08:35:27.813392 24089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:27.813439 24089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 075863553e9348e4b914efc0073f7382, State: Initialized, Role: FOLLOWER
I20260807 08:35:27.813930 24089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075863553e9348e4b914efc0073f7382" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42281 } }
I20260807 08:35:27.814061 24089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:35:27.814105 24089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:35:27.814184 24089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:27.815222 24089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075863553e9348e4b914efc0073f7382" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42281 } }
I20260807 08:35:27.815552 24089 leader_election.cc:302] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 075863553e9348e4b914efc0073f7382; no voters:
I20260807 08:35:27.815860 24089 leader_election.cc:288] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:35:27.816236 24096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:35:27.816588 24096 raft_consensus.cc:686] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [term 1 LEADER]: Becoming Leader. State: Replica: 075863553e9348e4b914efc0073f7382, State: Running, Role: LEADER
I20260807 08:35:27.817072 24096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075863553e9348e4b914efc0073f7382" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42281 } }
I20260807 08:35:27.817260 24089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:35:27.819692 24099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 075863553e9348e4b914efc0073f7382. Latest consensus state: current_term: 1 leader_uuid: "075863553e9348e4b914efc0073f7382" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075863553e9348e4b914efc0073f7382" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42281 } } }
I20260807 08:35:27.819878 24099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [sys.catalog]: This master's current role is: LEADER
I20260807 08:35:27.819671 24098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "075863553e9348e4b914efc0073f7382" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075863553e9348e4b914efc0073f7382" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42281 } } }
I20260807 08:35:27.821141 24098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382 [sys.catalog]: This master's current role is: LEADER
W20260807 08:35:27.822956 24114 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:35:27.823069 24114 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:35:27.823159 24115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:35:27.824265 24115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:35:27.831244 24115 catalog_manager.cc:1347] Generated new cluster ID: 50ba2e0cfdb34f48912ab9f787bf638a
I20260807 08:35:27.831396 24115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:35:27.839718 24115 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:35:27.840121 24115 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:35:27.850382 24115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 075863553e9348e4b914efc0073f7382: Generated new TSK 0
I20260807 08:35:27.851533 24115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:35:27.862586 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42281
--builtin_ntp_servers=127.9.174.20:43137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 08:35:28.031401 24121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:28.031898 24121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:28.031981 24121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:28.037138 24121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:28.037231 24121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:28.037338 24121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:35:28.043092 24121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:28.044719 24121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:28.046540 24121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:28.059192 24133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:28.059298 24129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:28.059255 24128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:28.059684 24121 server_base.cc:1034] running on GCE node
I20260807 08:35:28.060240 24121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:28.061075 24121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:28.062263 24121 hybrid_clock.cc:648] HybridClock initialized: now 1786091728062245 us; error 42 us; skew 500 ppm
I20260807 08:35:28.064435 24121 webserver.cc:460] Webserver started at http://127.9.174.1:39485/ using document root <none> and password file <none>
I20260807 08:35:28.065148 24121 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:28.065266 24121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:28.065538 24121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:28.067974 24121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "b275c7c448a441629cba6a76cf0dd7da"
format_stamp: "Formatted at 2026-08-07 08:35:28 on dist-test-slave-h3ft"
server_key: "9fd31282206a09366251b3afc64aaf31"
server_key_iv: "ca4ba78e6e05918d8002d11731556af0"
server_key_version: "encryptionkey@0"
I20260807 08:35:28.068625 24121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "b275c7c448a441629cba6a76cf0dd7da"
format_stamp: "Formatted at 2026-08-07 08:35:28 on dist-test-slave-h3ft"
server_key: "9fd31282206a09366251b3afc64aaf31"
server_key_iv: "ca4ba78e6e05918d8002d11731556af0"
server_key_version: "encryptionkey@0"
I20260807 08:35:28.073553 24121 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 08:35:28.076929 24139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:28.078615 24121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 08:35:28.078747 24121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "b275c7c448a441629cba6a76cf0dd7da"
format_stamp: "Formatted at 2026-08-07 08:35:28 on dist-test-slave-h3ft"
server_key: "9fd31282206a09366251b3afc64aaf31"
server_key_iv: "ca4ba78e6e05918d8002d11731556af0"
server_key_version: "encryptionkey@0"
I20260807 08:35:28.078917 24121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:28.084080 24121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:28.084780 24121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:28.084995 24121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:28.085600 24121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:35:28.086627 24121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:28.086702 24121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:28.086771 24121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:28.086815 24121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:28.097997 24121 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:32785
I20260807 08:35:28.098060 24310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:32785 every 8 connection(s)
I20260807 08:35:28.099133 24121 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:35:28.105568 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 24121
I20260807 08:35:28.105687 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:35:28.106019 9912 external_mini_cluster.cc:1428] Setting key b5f938a80a40231c487b9985ec60851b
I20260807 08:35:28.107300 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:28.101687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:36153 (local address 127.9.174.62:42281)
0807 08:35:28.101867 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:28.101872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:28.103288 (+ 1416us) server_negotiation.cc:407] Connection header received
0807 08:35:28.104961 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:28.104965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:28.105028 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:28.105123 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:28.106656 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:28.106660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:28.106663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:35:28.106679 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:28.106714 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:35:28.106724 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:28.106843 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:28.107055 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 08:35:28.107120 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 08:35:28.108708 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42281
--builtin_ntp_servers=127.9.174.20:43137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 08:35:28.108872 24313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:28.102070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42281 (local address 127.9.174.1:36153)
0807 08:35:28.103089 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:28.103149 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:28.104670 (+ 1521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:28.105366 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:28.105376 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:28.105500 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:28.106318 (+ 818us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:35:28.106320 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:28.106377 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:35:28.106468 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:28.106821 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:28.106831 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:28.106838 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 08:35:28.107101 (+ 263us) client_negotiation.cc:240] Negotiation successful
0807 08:35:28.107560 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":146,"threads_started":1}
I20260807 08:35:28.110473 24311 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42281
I20260807 08:35:28.110827 24311 heartbeater.cc:464] Registering TS with master...
I20260807 08:35:28.111554 24311 heartbeater.cc:511] Master 127.9.174.62:42281 requested a full tablet report, sending...
I20260807 08:35:28.113631 24031 ts_manager.cc:194] Registered new tserver with Master: b275c7c448a441629cba6a76cf0dd7da (127.9.174.1:32785)
W20260807 08:35:28.232555 24314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:28.232868 24314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:28.232909 24314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:28.236965 24314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:28.237028 24314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:28.237085 24314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:35:28.241742 24314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:28.242993 24314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:28.244525 24314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:28.256193 24330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:28.256448 24314 server_base.cc:1034] running on GCE node
W20260807 08:35:28.256193 24322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:28.256321 24323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:28.257138 24314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:28.258028 24314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:28.259264 24314 hybrid_clock.cc:648] HybridClock initialized: now 1786091728259214 us; error 64 us; skew 500 ppm
I20260807 08:35:28.261694 24314 webserver.cc:460] Webserver started at http://127.9.174.2:41255/ using document root <none> and password file <none>
I20260807 08:35:28.262502 24314 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:28.262609 24314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:28.262894 24314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:28.265763 24314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "0f944c1e2d6e4da0a7341c5b4d2d2a7a"
format_stamp: "Formatted at 2026-08-07 08:35:28 on dist-test-slave-h3ft"
server_key: "302cc577d96d585987c11539b14d98e0"
server_key_iv: "e80f6c0766f0e62fd38a0d33646cd745"
server_key_version: "encryptionkey@0"
I20260807 08:35:28.266503 24314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "0f944c1e2d6e4da0a7341c5b4d2d2a7a"
format_stamp: "Formatted at 2026-08-07 08:35:28 on dist-test-slave-h3ft"
server_key: "302cc577d96d585987c11539b14d98e0"
server_key_iv: "e80f6c0766f0e62fd38a0d33646cd745"
server_key_version: "encryptionkey@0"
I20260807 08:35:28.271574 24314 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 08:35:28.275350 24337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:28.277406 24314 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:35:28.277621 24314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "0f944c1e2d6e4da0a7341c5b4d2d2a7a"
format_stamp: "Formatted at 2026-08-07 08:35:28 on dist-test-slave-h3ft"
server_key: "302cc577d96d585987c11539b14d98e0"
server_key_iv: "e80f6c0766f0e62fd38a0d33646cd745"
server_key_version: "encryptionkey@0"
I20260807 08:35:28.277796 24314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:28.283953 24314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:28.284826 24314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:28.285066 24314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:28.285799 24314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:35:28.287091 24314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:28.287166 24314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:28.287246 24314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:28.287292 24314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:28.299454 24314 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:36227
I20260807 08:35:28.299541 24513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:36227 every 8 connection(s)
I20260807 08:35:28.300693 24314 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:35:28.308647 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:28.303301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:47725 (local address 127.9.174.62:42281)
0807 08:35:28.303524 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:28.303530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:28.304966 (+ 1436us) server_negotiation.cc:407] Connection header received
0807 08:35:28.306436 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:28.306440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:28.306511 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:28.306649 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:28.308072 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:28.308076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:28.308078 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:35:28.308097 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:28.308124 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:35:28.308133 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:28.308246 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:28.308421 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 08:35:28.308477 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 08:35:28.309557 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 24314
I20260807 08:35:28.309582 24517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:28.303881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42281 (local address 127.9.174.2:47725)
0807 08:35:28.304773 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:28.304825 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:28.306171 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:28.306849 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:28.306856 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:28.306959 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:28.307745 (+ 786us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:35:28.307747 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:28.307793 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:35:28.307865 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:28.308240 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:28.308248 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:28.308252 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:35:28.308411 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 08:35:28.308620 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":164,"threads_started":1}
I20260807 08:35:28.309729 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:35:28.310221 9912 external_mini_cluster.cc:1428] Setting key 1a06ef5df3477273adeb3f139b67b2ca
I20260807 08:35:28.311305 24514 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42281
I20260807 08:35:28.311601 24514 heartbeater.cc:464] Registering TS with master...
I20260807 08:35:28.312198 24514 heartbeater.cc:511] Master 127.9.174.62:42281 requested a full tablet report, sending...
I20260807 08:35:28.313174 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42281
--builtin_ntp_servers=127.9.174.20:43137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 08:35:28.313531 24031 ts_manager.cc:194] Registered new tserver with Master: 0f944c1e2d6e4da0a7341c5b4d2d2a7a (127.9.174.2:36227)
W20260807 08:35:28.456614 24519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:28.456961 24519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:28.457017 24519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:28.461236 24519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:28.461301 24519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:28.461364 24519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:35:28.466513 24519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:28.467969 24519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:28.469684 24519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:28.481570 24531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:28.481570 24534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:28.481617 24530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:28.481937 24519 server_base.cc:1034] running on GCE node
I20260807 08:35:28.482571 24519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:28.483573 24519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:28.484776 24519 hybrid_clock.cc:648] HybridClock initialized: now 1786091728484744 us; error 44 us; skew 500 ppm
I20260807 08:35:28.487252 24519 webserver.cc:460] Webserver started at http://127.9.174.3:42357/ using document root <none> and password file <none>
I20260807 08:35:28.487967 24519 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:28.488050 24519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:28.488261 24519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:28.490770 24519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "e393418f232c4f90993b979804bfa81d"
format_stamp: "Formatted at 2026-08-07 08:35:28 on dist-test-slave-h3ft"
server_key: "d23da5d7de2212e5c42e498f233931ab"
server_key_iv: "f6b01c6152ea160075bea9d6b6c58e6f"
server_key_version: "encryptionkey@0"
I20260807 08:35:28.491442 24519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "e393418f232c4f90993b979804bfa81d"
format_stamp: "Formatted at 2026-08-07 08:35:28 on dist-test-slave-h3ft"
server_key: "d23da5d7de2212e5c42e498f233931ab"
server_key_iv: "f6b01c6152ea160075bea9d6b6c58e6f"
server_key_version: "encryptionkey@0"
I20260807 08:35:28.495985 24519 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:35:28.499184 24542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:28.500823 24519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:35:28.501032 24519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "e393418f232c4f90993b979804bfa81d"
format_stamp: "Formatted at 2026-08-07 08:35:28 on dist-test-slave-h3ft"
server_key: "d23da5d7de2212e5c42e498f233931ab"
server_key_iv: "f6b01c6152ea160075bea9d6b6c58e6f"
server_key_version: "encryptionkey@0"
I20260807 08:35:28.501188 24519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:28.507395 24519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:28.508206 24519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:28.508451 24519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:28.509172 24519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:35:28.510414 24519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:28.510496 24519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:28.510579 24519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:28.510624 24519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:28.522751 24519 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:40321
I20260807 08:35:28.522776 24721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:40321 every 8 connection(s)
I20260807 08:35:28.524106 24519 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:35:28.531739 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:28.526516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:39939 (local address 127.9.174.62:42281)
0807 08:35:28.526705 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:28.526709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:28.527964 (+ 1255us) server_negotiation.cc:407] Connection header received
0807 08:35:28.529485 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:28.529488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:28.529539 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:28.529623 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:28.531123 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:28.531127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:28.531129 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:35:28.531142 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:28.531165 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:35:28.531174 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:28.531266 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:28.531492 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 08:35:28.531571 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 08:35:28.532832 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:28.526900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42281 (local address 127.9.174.3:39939)
0807 08:35:28.527761 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:28.527808 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:28.529251 (+ 1443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:28.529823 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:28.529831 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:28.529989 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:28.530713 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:35:28.530715 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:28.530765 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:35:28.530843 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:28.531289 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:28.531299 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:28.531305 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:35:28.531490 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 08:35:28.531748 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":140,"threads_started":1}
I20260807 08:35:28.534198 24722 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42281
I20260807 08:35:28.534375 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 24519
I20260807 08:35:28.534538 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:35:28.534574 24722 heartbeater.cc:464] Registering TS with master...
I20260807 08:35:28.534970 9912 external_mini_cluster.cc:1428] Setting key f8178ffdf40838cfee0463a509131b81
I20260807 08:35:28.535193 24722 heartbeater.cc:511] Master 127.9.174.62:42281 requested a full tablet report, sending...
I20260807 08:35:28.536495 24031 ts_manager.cc:194] Registered new tserver with Master: e393418f232c4f90993b979804bfa81d (127.9.174.3:40321)
I20260807 08:35:28.537451 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 08:35:28.541309 24092 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:35:28.540168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60720 (local address 127.9.174.62:42281)
0807 08:35:28.540322 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:28.540327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:28.540519 (+ 192us) server_negotiation.cc:407] Connection header received
0807 08:35:28.540682 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:28.540686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:28.540732 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:28.540820 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:28.541152 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60720: BlockingRecv error: recv got EOF from 127.0.0.1:60720 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 08:35:28.541442 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 24121
I20260807 08:35:28.559659 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 24314
I20260807 08:35:28.578482 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 24519
I20260807 08:35:28.596479 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 23989
2026-08-07T08:35:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (1056 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:35:28 dist-test-slave-h3ft krb5kdc[24734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:35:28 dist-test-slave-h3ft krb5kdc[24734](info): set up 2 sockets
Aug 07 08:35:28 dist-test-slave-h3ft krb5kdc[24734](info): commencing operation
krb5kdc: starting...
W20260807 08:35:29.880200 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:35:29 dist-test-slave-h3ft krb5kdc[24734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:35:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:35:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:35:30.042223 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:35315
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:35315
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:35:30.181026 24756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:30.181517 24756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:30.181617 24756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:30.186587 24756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:35:30.186787 24756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:30.186833 24756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:30.186880 24756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:35:30.186908 24756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:35:30.194456 24756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:35315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:35315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:30.196174 24756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:30.197537 24756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:30.208421 24770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:30.208436 24767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:30.208483 24768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:30.208628 24756 server_base.cc:1034] running on GCE node
I20260807 08:35:30.209636 24756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:30.211195 24756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:30.212409 24756 hybrid_clock.cc:648] HybridClock initialized: now 1786091730212361 us; error 66 us; skew 500 ppm
Aug 07 08:35:30 dist-test-slave-h3ft krb5kdc[24734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091730, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:35:30.220933 24756 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:35:30.222378 24756 webserver.cc:460] Webserver started at http://127.9.174.62:36647/ using document root <none> and password file <none>
I20260807 08:35:30.223209 24756 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:30.223320 24756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:30.223614 24756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:30.226514 24756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "f5cd6c88e61043088212610f7824d1f5"
format_stamp: "Formatted at 2026-08-07 08:35:30 on dist-test-slave-h3ft"
server_key: "81e2aabad018107da4a9211a353bff35"
server_key_iv: "f2b32f393a6092f2e6ead171df320222"
server_key_version: "encryptionkey@0"
I20260807 08:35:30.227466 24756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "f5cd6c88e61043088212610f7824d1f5"
format_stamp: "Formatted at 2026-08-07 08:35:30 on dist-test-slave-h3ft"
server_key: "81e2aabad018107da4a9211a353bff35"
server_key_iv: "f2b32f393a6092f2e6ead171df320222"
server_key_version: "encryptionkey@0"
I20260807 08:35:30.233042 24756 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 08:35:30.236829 24786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:30.238710 24756 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:35:30.238955 24756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "f5cd6c88e61043088212610f7824d1f5"
format_stamp: "Formatted at 2026-08-07 08:35:30 on dist-test-slave-h3ft"
server_key: "81e2aabad018107da4a9211a353bff35"
server_key_iv: "f2b32f393a6092f2e6ead171df320222"
server_key_version: "encryptionkey@0"
I20260807 08:35:30.239220 24756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:30.262215 24756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:30.266566 24756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:30.266844 24756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:30.276746 24756 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:35315
I20260807 08:35:30.277103 24869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:35315 every 8 connection(s)
I20260807 08:35:30.278126 24756 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:35:30.281980 24871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:30.283152 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 24756
I20260807 08:35:30.283299 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:35:30.283653 9912 external_mini_cluster.cc:1428] Setting key abc88090fa323a578e830b301f11d51f
I20260807 08:35:30.289661 24871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5: Bootstrap starting.
I20260807 08:35:30.292476 24871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:30.293443 24871 log.cc:824] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:35:30 dist-test-slave-h3ft krb5kdc[24734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:35:30.296200 24871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5: No bootstrap required, opened a new log
I20260807 08:35:30.300314 24871 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5cd6c88e61043088212610f7824d1f5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35315 } }
I20260807 08:35:30.300601 24871 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:30.300694 24871 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5cd6c88e61043088212610f7824d1f5, State: Initialized, Role: FOLLOWER
I20260807 08:35:30.301223 24871 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5cd6c88e61043088212610f7824d1f5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35315 } }
I20260807 08:35:30.301438 24871 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:35:30.301528 24871 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:35:30.301642 24871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:30.302709 24871 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5cd6c88e61043088212610f7824d1f5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35315 } }
I20260807 08:35:30.302752 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:30.285752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44764 (local address 127.9.174.62:35315)
0807 08:35:30.286367 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:30.286399 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:30.286453 (+ 54us) server_negotiation.cc:407] Connection header received
0807 08:35:30.287599 (+ 1146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:30.287628 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:30.288101 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:30.288520 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:30.289549 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:30.290865 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:30.292187 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:30.292907 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:30.293083 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:30.296365 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:30.296374 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:30.296402 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:30.296450 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:30.298739 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:30.299439 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:30.299447 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:30.299455 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:30.299598 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:30.299933 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:30.299947 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:30.299949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:30.300488 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:30.300761 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:30.301087 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 08:35:30.301557 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":192,"threads_started":1}
I20260807 08:35:30.303128 24871 leader_election.cc:302] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5cd6c88e61043088212610f7824d1f5; no voters:
I20260807 08:35:30.303519 24871 leader_election.cc:288] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:35:30.303701 24876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:35:30.304081 24876 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [term 1 LEADER]: Becoming Leader. State: Replica: f5cd6c88e61043088212610f7824d1f5, State: Running, Role: LEADER
I20260807 08:35:30.304550 24876 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5cd6c88e61043088212610f7824d1f5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35315 } }
I20260807 08:35:30.305100 24871 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:35:30.306785 24880 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5cd6c88e61043088212610f7824d1f5. Latest consensus state: current_term: 1 leader_uuid: "f5cd6c88e61043088212610f7824d1f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5cd6c88e61043088212610f7824d1f5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35315 } } }
I20260807 08:35:30.306969 24880 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:35:30.307404 24878 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5cd6c88e61043088212610f7824d1f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5cd6c88e61043088212610f7824d1f5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35315 } } }
I20260807 08:35:30.307564 24878 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:35:30.307493 24902 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:35:30.311537 24902 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:35:30.317555 24902 catalog_manager.cc:1347] Generated new cluster ID: b15fb6f9ddad4f23a860513dc4d7939c
I20260807 08:35:30.317641 24902 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:35:30.329035 24902 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:35:30.330706 24902 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:35:30.337741 24902 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5: Generated new TSK 0
I20260807 08:35:30.338795 24902 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:35:30.374888 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:35315
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:35:30.514871 24915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:30.515359 24915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:30.515439 24915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:30.519968 24915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:30.520026 24915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:30.520128 24915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:35:30.526365 24915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:35315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:30.527967 24915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:30.529423 24915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:30.541769 24923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:30.541769 24927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:30.541836 24925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:30.542119 24915 server_base.cc:1034] running on GCE node
I20260807 08:35:30.542719 24915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:30.543689 24915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:30.544888 24915 hybrid_clock.cc:648] HybridClock initialized: now 1786091730544874 us; error 38 us; skew 500 ppm
Aug 07 08:35:30 dist-test-slave-h3ft krb5kdc[24734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091730, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:35:30.548259 24915 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:35:30.549353 24915 webserver.cc:460] Webserver started at http://127.9.174.1:43545/ using document root <none> and password file <none>
I20260807 08:35:30.550064 24915 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:30.550166 24915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:30.550446 24915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:30.553144 24915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "a38d1b2e0ec24c2f8a607db769e540a9"
format_stamp: "Formatted at 2026-08-07 08:35:30 on dist-test-slave-h3ft"
server_key: "62bd31a65ffaf1b52109cdff6007e8c0"
server_key_iv: "639dfac690d044b1f41cc5f0a072f280"
server_key_version: "encryptionkey@0"
I20260807 08:35:30.553877 24915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "a38d1b2e0ec24c2f8a607db769e540a9"
format_stamp: "Formatted at 2026-08-07 08:35:30 on dist-test-slave-h3ft"
server_key: "62bd31a65ffaf1b52109cdff6007e8c0"
server_key_iv: "639dfac690d044b1f41cc5f0a072f280"
server_key_version: "encryptionkey@0"
I20260807 08:35:30.558650 24915 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 08:35:30.561924 24941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:30.563832 24915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 08:35:30.564031 24915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "a38d1b2e0ec24c2f8a607db769e540a9"
format_stamp: "Formatted at 2026-08-07 08:35:30 on dist-test-slave-h3ft"
server_key: "62bd31a65ffaf1b52109cdff6007e8c0"
server_key_iv: "639dfac690d044b1f41cc5f0a072f280"
server_key_version: "encryptionkey@0"
I20260807 08:35:30.564210 24915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:30.606101 24915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:30.610153 24915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:30.610359 24915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:30.611160 24915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:35:30.612236 24915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:30.612304 24915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:30.612355 24915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:30.612370 24915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:30.623767 24915 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:34017
I20260807 08:35:30.623795 25114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:34017 every 8 connection(s)
I20260807 08:35:30.625016 24915 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:35:30.635298 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 24915
I20260807 08:35:30.635497 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:35:30.635955 9912 external_mini_cluster.cc:1428] Setting key 48971b8c75d0db9f0b23e7d54a2dc2ea
Aug 07 08:35:30 dist-test-slave-h3ft krb5kdc[24734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091730, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:35:30.642668 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:30.627478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:60503 (local address 127.9.174.62:35315)
0807 08:35:30.627703 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:30.627708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:30.628821 (+ 1113us) server_negotiation.cc:407] Connection header received
0807 08:35:30.630636 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:30.630640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:30.630715 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:30.630839 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:30.632467 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:30.633087 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:30.634437 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:30.634697 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:30.634817 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:30.638373 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:30.638377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:30.638380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:30.638405 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:30.640096 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:30.640722 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:30.640727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:30.640729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:30.640822 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:30.641262 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:30.641265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:30.641267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:30.641504 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:30.641623 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:30.642287 (+ 664us) server_negotiation.cc:299] Negotiation successful
0807 08:35:30.642460 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 08:35:30.643981 25120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:30.628003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:35315 (local address 127.9.174.1:60503)
0807 08:35:30.628609 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:30.628665 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:30.630339 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:30.631129 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:30.631139 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:30.631645 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:30.632286 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:30.632303 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:30.633245 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:30.633249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:30.634292 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:30.634302 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:30.634911 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:30.634916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:30.635215 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:30.636112 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:30.636134 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:30.638155 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:30.640235 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:30.640242 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:30.640266 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:30.640606 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:30.640972 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:30.640975 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:30.640976 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:30.641151 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:30.641621 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:30.641631 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:30.641998 (+ 367us) client_negotiation.cc:769] Sending connection context
0807 08:35:30.642245 (+ 247us) client_negotiation.cc:240] Negotiation successful
0807 08:35:30.642700 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":157,"threads_started":1}
I20260807 08:35:30.645637 25115 heartbeater.cc:347] Connected to a master server at 127.9.174.62:35315
I20260807 08:35:30.646034 25115 heartbeater.cc:464] Registering TS with master...
I20260807 08:35:30.646916 25115 heartbeater.cc:511] Master 127.9.174.62:35315 requested a full tablet report, sending...
I20260807 08:35:30.649333 24818 ts_manager.cc:194] Registered new tserver with Master: a38d1b2e0ec24c2f8a607db769e540a9 (127.9.174.1:34017)
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
I20260807 08:35:30.650935 24818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:60503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:35:30.674787 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:35315
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:35:30.820041 25124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:30.820401 25124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:30.820451 25124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:30.824918 25124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:30.825017 25124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:30.825088 25124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:35:30.831378 25124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:35315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:30.832914 25124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:30.834183 25124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:35:30.848765 25136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:30.848729 25130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:30.849538 25132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:30.850517 25124 server_base.cc:1034] running on GCE node
I20260807 08:35:30.851401 25124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:30.852394 25124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:30.853653 25124 hybrid_clock.cc:648] HybridClock initialized: now 1786091730853607 us; error 57 us; skew 500 ppm
Aug 07 08:35:30 dist-test-slave-h3ft krb5kdc[24734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091730, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:35:30.857932 25124 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:35:30.859551 25124 webserver.cc:460] Webserver started at http://127.9.174.2:43315/ using document root <none> and password file <none>
I20260807 08:35:30.860472 25124 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:30.860594 25124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:30.860913 25124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:30.863682 25124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "be714da5fe64440ba59c825ee5040eda"
format_stamp: "Formatted at 2026-08-07 08:35:30 on dist-test-slave-h3ft"
server_key: "4cc42a51ce3f73ed313736933e47a257"
server_key_iv: "aa7f595b39f8b030e5035aca0a6f3f49"
server_key_version: "encryptionkey@0"
I20260807 08:35:30.864488 25124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "be714da5fe64440ba59c825ee5040eda"
format_stamp: "Formatted at 2026-08-07 08:35:30 on dist-test-slave-h3ft"
server_key: "4cc42a51ce3f73ed313736933e47a257"
server_key_iv: "aa7f595b39f8b030e5035aca0a6f3f49"
server_key_version: "encryptionkey@0"
I20260807 08:35:30.869725 25124 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:35:30.873083 25145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:30.874827 25124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:35:30.875079 25124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "be714da5fe64440ba59c825ee5040eda"
format_stamp: "Formatted at 2026-08-07 08:35:30 on dist-test-slave-h3ft"
server_key: "4cc42a51ce3f73ed313736933e47a257"
server_key_iv: "aa7f595b39f8b030e5035aca0a6f3f49"
server_key_version: "encryptionkey@0"
I20260807 08:35:30.875237 25124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:30.888372 25124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:30.893055 25124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:30.893361 25124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:30.894237 25124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:35:30.895480 25124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:30.895560 25124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:30.895640 25124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:30.895694 25124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:30.907263 25124 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:44807
I20260807 08:35:30.907351 25320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:44807 every 8 connection(s)
I20260807 08:35:30.908470 25124 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:35:30.915549 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 25124
I20260807 08:35:30.915727 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:35:30.916116 9912 external_mini_cluster.cc:1428] Setting key 66ee007be41559c71b1d1cb9146d887d
Aug 07 08:35:30 dist-test-slave-h3ft krb5kdc[24734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091730, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:35:30.927635 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:30.910936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:35355 (local address 127.9.174.62:35315)
0807 08:35:30.911146 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:30.911151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:30.912378 (+ 1227us) server_negotiation.cc:407] Connection header received
0807 08:35:30.913994 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:30.913998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:30.914066 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:30.914194 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:30.915881 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:30.916551 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:30.918302 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:30.918558 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:30.918648 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:30.922472 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:30.922479 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:30.922484 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:30.922522 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:30.924566 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:30.925327 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:30.925333 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:30.925337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:30.925447 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:30.925828 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:30.925835 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:30.925840 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:30.926185 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:30.926312 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:30.927186 (+ 874us) server_negotiation.cc:299] Negotiation successful
0807 08:35:30.927376 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 08:35:30.928654 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:30.911564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:35315 (local address 127.9.174.2:35355)
0807 08:35:30.912175 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:30.912227 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:30.913714 (+ 1487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:30.914397 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:30.914405 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:30.914864 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:30.915630 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:30.915647 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:30.916736 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:30.916739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:30.918145 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:30.918161 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:30.918745 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:30.918751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:30.919050 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:30.919850 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:30.919892 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:30.922244 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:30.924726 (+ 2482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:30.924732 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:30.924751 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:30.925185 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:30.925548 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:30.925550 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:30.925551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:30.925712 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:30.926325 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:30.926332 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:30.926829 (+ 497us) client_negotiation.cc:769] Sending connection context
0807 08:35:30.927105 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 08:35:30.927470 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":158,"threads_started":1}
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
I20260807 08:35:30.930302 25321 heartbeater.cc:347] Connected to a master server at 127.9.174.62:35315
I20260807 08:35:30.930686 25321 heartbeater.cc:464] Registering TS with master...
I20260807 08:35:30.931520 25321 heartbeater.cc:511] Master 127.9.174.62:35315 requested a full tablet report, sending...
I20260807 08:35:30.933116 24818 ts_manager.cc:194] Registered new tserver with Master: be714da5fe64440ba59c825ee5040eda (127.9.174.2:44807)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
I20260807 08:35:30.933971 24818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:35355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:35:30.955960 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:35315
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:35:31.099270 25328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:35:31.099701 25328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:35:31.099795 25328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:35:31.104313 25328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:35:31.104457 25328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:35:31.104583 25328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:35:31.110954 25328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:35315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:35:31.112419 25328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:35:31.113703 25328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:35:31.126000 25328 server_base.cc:1034] running on GCE node
W20260807 08:35:31.125837 25336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:31.125828 25337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:35:31.125828 25340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:35:31.126968 25328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:35:31.127799 25328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:35:31.129024 25328 hybrid_clock.cc:648] HybridClock initialized: now 1786091731128984 us; error 59 us; skew 500 ppm
Aug 07 08:35:31 dist-test-slave-h3ft krb5kdc[24734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091731, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:35:31.132714 25328 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:35:31.133963 25328 webserver.cc:460] Webserver started at http://127.9.174.3:40565/ using document root <none> and password file <none>
I20260807 08:35:31.134691 25328 fs_manager.cc:359] Metadata directory not provided
I20260807 08:35:31.134791 25328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:35:31.135164 25328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:35:31.137804 25328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "0ba6cde74457404fa3dd988f7222adf6"
format_stamp: "Formatted at 2026-08-07 08:35:31 on dist-test-slave-h3ft"
server_key: "38ddce6d953f4b2f60ed092aaae1cfd7"
server_key_iv: "9e9f0b0f59be6c8dc2f1e1424e9e9b11"
server_key_version: "encryptionkey@0"
I20260807 08:35:31.138497 25328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "0ba6cde74457404fa3dd988f7222adf6"
format_stamp: "Formatted at 2026-08-07 08:35:31 on dist-test-slave-h3ft"
server_key: "38ddce6d953f4b2f60ed092aaae1cfd7"
server_key_iv: "9e9f0b0f59be6c8dc2f1e1424e9e9b11"
server_key_version: "encryptionkey@0"
I20260807 08:35:31.143417 25328 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 08:35:31.146592 25354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:31.148195 25328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:35:31.148371 25328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "0ba6cde74457404fa3dd988f7222adf6"
format_stamp: "Formatted at 2026-08-07 08:35:31 on dist-test-slave-h3ft"
server_key: "38ddce6d953f4b2f60ed092aaae1cfd7"
server_key_iv: "9e9f0b0f59be6c8dc2f1e1424e9e9b11"
server_key_version: "encryptionkey@0"
I20260807 08:35:31.148518 25328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:35:31.165791 25328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:35:31.169598 25328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:35:31.169836 25328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:35:31.170665 25328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:35:31.171710 25328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:35:31.171783 25328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:31.171867 25328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:35:31.171953 25328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:35:31.183176 25328 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:40199
I20260807 08:35:31.183200 25528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:40199 every 8 connection(s)
I20260807 08:35:31.184335 25328 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:35:31.186300 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 25328
I20260807 08:35:31.186486 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:35:31.186915 9912 external_mini_cluster.cc:1428] Setting key 12f7e447bf1561054ac7230080cbe5fd
Aug 07 08:35:31 dist-test-slave-h3ft krb5kdc[24734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091731, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:35:31.202234 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:31.186801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:53601 (local address 127.9.174.62:35315)
0807 08:35:31.187040 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:31.187046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:31.188353 (+ 1307us) server_negotiation.cc:407] Connection header received
0807 08:35:31.190288 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:31.190292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:31.190348 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:31.190430 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:31.192073 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.192681 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.194063 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.194298 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.194385 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:31.197610 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:31.197613 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:31.197616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:31.197644 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:31.199267 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:31.199919 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:31.199923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:31.199924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:31.200022 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:31.200539 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:31.200546 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:31.200551 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:31.200828 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:31.200954 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:31.201838 (+ 884us) server_negotiation.cc:299] Negotiation successful
0807 08:35:31.202006 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 08:35:31.203428 25539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:31.187429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:35315 (local address 127.9.174.3:53601)
0807 08:35:31.188158 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:31.188233 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:31.190039 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:31.190649 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:31.190658 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:31.191259 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:35:31.191910 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:31.191927 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.192821 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.192824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:31.193935 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:31.193947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.194468 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.194471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:31.194650 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:31.195491 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:35:31.195511 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:35:31.197403 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:35:31.199415 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:31.199429 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:31.199445 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:31.199740 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:31.200155 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:35:31.200159 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:35:31.200161 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:35:31.200393 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:35:31.200988 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:35:31.200994 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:35:31.201478 (+ 484us) client_negotiation.cc:769] Sending connection context
0807 08:35:31.201731 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 08:35:31.202144 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":163,"threads_started":1}
I20260807 08:35:31.204756 25534 heartbeater.cc:347] Connected to a master server at 127.9.174.62:35315
I20260807 08:35:31.205106 25534 heartbeater.cc:464] Registering TS with master...
I20260807 08:35:31.205785 25534 heartbeater.cc:511] Master 127.9.174.62:35315 requested a full tablet report, sending...
I20260807 08:35:31.207221 24818 ts_manager.cc:194] Registered new tserver with Master: 0ba6cde74457404fa3dd988f7222adf6 (127.9.174.3:40199)
I20260807 08:35:31.207898 24818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:53601
I20260807 08:35:31.214702 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:35:31.225795 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:31.217283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44772 (local address 127.9.174.62:35315)
0807 08:35:31.217440 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:31.217444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:31.217641 (+ 197us) server_negotiation.cc:407] Connection header received
0807 08:35:31.217759 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:31.217763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:31.217814 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:31.217937 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:35:31.218798 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.219450 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.220363 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.220582 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.220665 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:31.222208 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:35:31.222211 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:35:31.222213 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:35:31.222237 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:35:31.223920 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:31.224405 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:31.224408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:31.224410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:31.224503 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:35:31.224807 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:35:31.224810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:35:31.224811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:35:31.225122 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:35:31.225247 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:31.225509 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 08:35:31.225625 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 08:35:31.229499 24818 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:35:31.233693 24818 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:35:31.251874 25554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:31.245747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54598 (local address 127.9.174.1:34017)
0807 08:35:31.246231 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:31.246240 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:31.246269 (+ 29us) server_negotiation.cc:407] Connection header received
0807 08:35:31.246369 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:31.246374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:31.246577 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:31.246756 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:31.247150 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.248342 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.249927 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.250736 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.250856 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:31.251081 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:31.251561 (+ 480us) server_negotiation.cc:299] Negotiation successful
0807 08:35:31.251693 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":100,"threads_started":1}
I20260807 08:35:31.252025 25555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:31.246114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:40199 (local address 127.0.0.1:37634)
0807 08:35:31.246525 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:31.246539 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:31.246639 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:31.247928 (+ 1289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:31.247931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:31.247949 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:31.248035 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:31.248040 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.249336 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.249341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:31.250235 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:31.250246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.251640 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.251643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:31.251733 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:31.251757 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 08:35:31.251828 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 08:35:31.251918 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":119,"threads_started":1}
I20260807 08:35:31.251854 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:31.244915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:44807 (local address 127.0.0.1:53800)
0807 08:35:31.245640 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:31.245702 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:31.245862 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:31.247160 (+ 1298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:31.247163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:31.247181 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:31.247294 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:31.247299 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.248984 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.248988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:31.250054 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:31.250061 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.251232 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.251236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:31.251350 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:31.251404 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 08:35:31.251453 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 08:35:31.251644 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":3,"thread_start_us":107,"threads_started":1}
I20260807 08:35:31.252440 25553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:31.245350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53800 (local address 127.9.174.2:44807)
0807 08:35:31.245985 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:31.246003 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:31.246035 (+ 32us) server_negotiation.cc:407] Connection header received
0807 08:35:31.246133 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:31.246139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:31.246403 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:31.247444 (+ 1041us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:31.247653 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.248813 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.250183 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.251077 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.251928 (+ 851us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:31.252086 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:31.252145 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 08:35:31.252244 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":186,"threads_started":1}
I20260807 08:35:31.251849 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:31.244942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:34017 (local address 127.0.0.1:54598)
0807 08:35:31.246140 (+ 1198us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:31.246153 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:31.246240 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:31.246752 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:31.246759 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:31.246786 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:31.246979 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:31.246988 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.248475 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.248477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:31.249786 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:31.249797 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.251141 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.251145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:31.251274 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:31.251328 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 08:35:31.251520 (+ 192us) client_negotiation.cc:240] Negotiation successful
0807 08:35:31.251675 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1137,"mutex_wait_us":51,"thread_start_us":131,"threads_started":1}
I20260807 08:35:31.253670 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:31.246642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37634 (local address 127.9.174.3:40199)
0807 08:35:31.247210 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:31.247223 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:31.247249 (+ 26us) server_negotiation.cc:407] Connection header received
0807 08:35:31.247344 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:31.247349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:31.247571 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:31.247741 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:31.248175 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.249204 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.250364 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.251317 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.251418 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:31.251573 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:31.252419 (+ 846us) server_negotiation.cc:299] Negotiation successful
0807 08:35:31.252536 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":235,"threads_started":1}
I20260807 08:35:31.256062 25017 tablet_service.cc:1467] Processing CreateTablet for tablet c33f85cb90f4467ba4747ffe9e2de4e8 (DEFAULT_TABLE table=test-table [id=d08ac9d44d1c49cdb41ec7aed881ca7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:31.256084 25211 tablet_service.cc:1467] Processing CreateTablet for tablet c33f85cb90f4467ba4747ffe9e2de4e8 (DEFAULT_TABLE table=test-table [id=d08ac9d44d1c49cdb41ec7aed881ca7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:31.256366 25430 tablet_service.cc:1467] Processing CreateTablet for tablet c33f85cb90f4467ba4747ffe9e2de4e8 (DEFAULT_TABLE table=test-table [id=d08ac9d44d1c49cdb41ec7aed881ca7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:35:31.257299 25017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c33f85cb90f4467ba4747ffe9e2de4e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:31.257306 25211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c33f85cb90f4467ba4747ffe9e2de4e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:31.257313 25430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c33f85cb90f4467ba4747ffe9e2de4e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:35:31.264742 25559 tablet_bootstrap.cc:492] T c33f85cb90f4467ba4747ffe9e2de4e8 P a38d1b2e0ec24c2f8a607db769e540a9: Bootstrap starting.
I20260807 08:35:31.266268 25562 tablet_bootstrap.cc:492] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda: Bootstrap starting.
I20260807 08:35:31.266368 25561 tablet_bootstrap.cc:492] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6: Bootstrap starting.
I20260807 08:35:31.268292 25559 tablet_bootstrap.cc:654] T c33f85cb90f4467ba4747ffe9e2de4e8 P a38d1b2e0ec24c2f8a607db769e540a9: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:31.268667 25561 tablet_bootstrap.cc:654] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:31.269264 25562 tablet_bootstrap.cc:654] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda: Neither blocks nor log segments found. Creating new log.
I20260807 08:35:31.269328 25559 log.cc:824] T c33f85cb90f4467ba4747ffe9e2de4e8 P a38d1b2e0ec24c2f8a607db769e540a9: Log is configured to *not* fsync() on all Append() calls
I20260807 08:35:31.270185 25562 log.cc:824] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda: Log is configured to *not* fsync() on all Append() calls
I20260807 08:35:31.270185 25561 log.cc:824] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6: Log is configured to *not* fsync() on all Append() calls
I20260807 08:35:31.272004 25559 tablet_bootstrap.cc:492] T c33f85cb90f4467ba4747ffe9e2de4e8 P a38d1b2e0ec24c2f8a607db769e540a9: No bootstrap required, opened a new log
I20260807 08:35:31.272210 25562 tablet_bootstrap.cc:492] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda: No bootstrap required, opened a new log
I20260807 08:35:31.272277 25559 ts_tablet_manager.cc:1397] T c33f85cb90f4467ba4747ffe9e2de4e8 P a38d1b2e0ec24c2f8a607db769e540a9: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 08:35:31.272425 25562 ts_tablet_manager.cc:1397] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 08:35:31.276453 25562 raft_consensus.cc:348] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be714da5fe64440ba59c825ee5040eda" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44807 } } peers { permanent_uuid: "a38d1b2e0ec24c2f8a607db769e540a9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 34017 } } peers { permanent_uuid: "0ba6cde74457404fa3dd988f7222adf6" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40199 } }
I20260807 08:35:31.276814 25562 raft_consensus.cc:374] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:31.276901 25562 raft_consensus.cc:729] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be714da5fe64440ba59c825ee5040eda, State: Initialized, Role: FOLLOWER
I20260807 08:35:31.276717 25559 raft_consensus.cc:348] T c33f85cb90f4467ba4747ffe9e2de4e8 P a38d1b2e0ec24c2f8a607db769e540a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be714da5fe64440ba59c825ee5040eda" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44807 } } peers { permanent_uuid: "a38d1b2e0ec24c2f8a607db769e540a9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 34017 } } peers { permanent_uuid: "0ba6cde74457404fa3dd988f7222adf6" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40199 } }
I20260807 08:35:31.276983 25559 raft_consensus.cc:374] T c33f85cb90f4467ba4747ffe9e2de4e8 P a38d1b2e0ec24c2f8a607db769e540a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:31.277037 25559 raft_consensus.cc:729] T c33f85cb90f4467ba4747ffe9e2de4e8 P a38d1b2e0ec24c2f8a607db769e540a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38d1b2e0ec24c2f8a607db769e540a9, State: Initialized, Role: FOLLOWER
I20260807 08:35:31.277547 25559 consensus_queue.cc:260] T c33f85cb90f4467ba4747ffe9e2de4e8 P a38d1b2e0ec24c2f8a607db769e540a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be714da5fe64440ba59c825ee5040eda" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44807 } } peers { permanent_uuid: "a38d1b2e0ec24c2f8a607db769e540a9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 34017 } } peers { permanent_uuid: "0ba6cde74457404fa3dd988f7222adf6" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40199 } }
I20260807 08:35:31.277547 25562 consensus_queue.cc:260] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be714da5fe64440ba59c825ee5040eda" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44807 } } peers { permanent_uuid: "a38d1b2e0ec24c2f8a607db769e540a9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 34017 } } peers { permanent_uuid: "0ba6cde74457404fa3dd988f7222adf6" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40199 } }
I20260807 08:35:31.278061 25561 tablet_bootstrap.cc:492] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6: No bootstrap required, opened a new log
I20260807 08:35:31.278348 25561 ts_tablet_manager.cc:1397] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260807 08:35:31.278777 25559 ts_tablet_manager.cc:1428] T c33f85cb90f4467ba4747ffe9e2de4e8 P a38d1b2e0ec24c2f8a607db769e540a9: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 08:35:31.278779 25562 ts_tablet_manager.cc:1428] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 08:35:31.279273 25115 heartbeater.cc:503] Master 127.9.174.62:35315 was elected leader, sending a full tablet report...
I20260807 08:35:31.279937 25321 heartbeater.cc:503] Master 127.9.174.62:35315 was elected leader, sending a full tablet report...
I20260807 08:35:31.284133 25561 raft_consensus.cc:348] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be714da5fe64440ba59c825ee5040eda" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44807 } } peers { permanent_uuid: "a38d1b2e0ec24c2f8a607db769e540a9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 34017 } } peers { permanent_uuid: "0ba6cde74457404fa3dd988f7222adf6" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40199 } }
I20260807 08:35:31.284456 25561 raft_consensus.cc:374] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:35:31.284518 25561 raft_consensus.cc:729] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ba6cde74457404fa3dd988f7222adf6, State: Initialized, Role: FOLLOWER
I20260807 08:35:31.285106 25561 consensus_queue.cc:260] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be714da5fe64440ba59c825ee5040eda" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44807 } } peers { permanent_uuid: "a38d1b2e0ec24c2f8a607db769e540a9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 34017 } } peers { permanent_uuid: "0ba6cde74457404fa3dd988f7222adf6" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40199 } }
I20260807 08:35:31.286336 25561 ts_tablet_manager.cc:1428] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 08:35:31.286782 25534 heartbeater.cc:503] Master 127.9.174.62:35315 was elected leader, sending a full tablet report...
I20260807 08:35:31.366930 25571 raft_consensus.cc:482] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:35:31.367319 25571 raft_consensus.cc:504] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be714da5fe64440ba59c825ee5040eda" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44807 } } peers { permanent_uuid: "a38d1b2e0ec24c2f8a607db769e540a9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 34017 } } peers { permanent_uuid: "0ba6cde74457404fa3dd988f7222adf6" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40199 } }
I20260807 08:35:31.369210 25571 leader_election.cc:288] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be714da5fe64440ba59c825ee5040eda (127.9.174.2:44807), a38d1b2e0ec24c2f8a607db769e540a9 (127.9.174.1:34017)
I20260807 08:35:31.373557 25539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:31.369639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:34017 (local address 127.9.174.3:46355)
0807 08:35:31.369828 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:31.369849 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:31.369971 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:31.370312 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:31.370317 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:31.370343 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:31.370510 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:31.370521 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.371415 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.371419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:31.372286 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:31.372295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.373043 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.373047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:31.373163 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:31.373192 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 08:35:31.373288 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 08:35:31.373380 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 08:35:31.373557 25554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:31.369774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:46355 (local address 127.9.174.1:34017)
0807 08:35:31.369988 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:31.369997 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:31.370027 (+ 30us) server_negotiation.cc:407] Connection header received
0807 08:35:31.370107 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:31.370110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:31.370178 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:31.370263 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:31.370660 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.371288 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.372431 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.372906 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.373031 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:31.373114 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:31.373307 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 08:35:31.373409 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:35:31.374023 25553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:31.369813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:35387 (local address 127.9.174.2:44807)
0807 08:35:31.370044 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:31.370049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:31.370461 (+ 412us) server_negotiation.cc:407] Connection header received
0807 08:35:31.370536 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:31.370539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:31.370592 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:31.370715 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:35:31.371064 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.371589 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.372827 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.373274 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.373443 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:31.373511 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:31.373796 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 08:35:31.373886 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260807 08:35:31.374485 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:31.369639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:44807 (local address 127.9.174.3:35387)
0807 08:35:31.370312 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:31.370326 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:35:31.370443 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:35:31.370728 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:35:31.370732 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:35:31.370749 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:35:31.370896 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:31.370904 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.371715 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.371720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:31.372693 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:35:31.372704 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.373450 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.373454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:35:31.373596 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:31.373615 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 08:35:31.374188 (+ 573us) client_negotiation.cc:240] Negotiation successful
0807 08:35:31.374353 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":188,"thread_start_us":175,"threads_started":1}
I20260807 08:35:31.374460 25044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c33f85cb90f4467ba4747ffe9e2de4e8" candidate_uuid: "0ba6cde74457404fa3dd988f7222adf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38d1b2e0ec24c2f8a607db769e540a9" is_pre_election: true
I20260807 08:35:31.374923 25044 raft_consensus.cc:2454] T c33f85cb90f4467ba4747ffe9e2de4e8 P a38d1b2e0ec24c2f8a607db769e540a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ba6cde74457404fa3dd988f7222adf6 in term 0.
I20260807 08:35:31.375260 25242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c33f85cb90f4467ba4747ffe9e2de4e8" candidate_uuid: "0ba6cde74457404fa3dd988f7222adf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be714da5fe64440ba59c825ee5040eda" is_pre_election: true
I20260807 08:35:31.375578 25357 leader_election.cc:302] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ba6cde74457404fa3dd988f7222adf6, a38d1b2e0ec24c2f8a607db769e540a9; no voters:
I20260807 08:35:31.375650 25242 raft_consensus.cc:2454] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ba6cde74457404fa3dd988f7222adf6 in term 0.
I20260807 08:35:31.375952 25571 raft_consensus.cc:2781] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:35:31.376071 25571 raft_consensus.cc:482] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:35:31.376142 25571 raft_consensus.cc:3037] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [term 0 FOLLOWER]: Advancing to term 1
W20260807 08:35:31.376657 25119 tablet.cc:2369] T c33f85cb90f4467ba4747ffe9e2de4e8 P a38d1b2e0ec24c2f8a607db769e540a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:35:31.377562 25571 raft_consensus.cc:504] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be714da5fe64440ba59c825ee5040eda" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44807 } } peers { permanent_uuid: "a38d1b2e0ec24c2f8a607db769e540a9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 34017 } } peers { permanent_uuid: "0ba6cde74457404fa3dd988f7222adf6" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40199 } }
I20260807 08:35:31.377975 25571 leader_election.cc:288] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [CANDIDATE]: Term 1 election: Requested vote from peers be714da5fe64440ba59c825ee5040eda (127.9.174.2:44807), a38d1b2e0ec24c2f8a607db769e540a9 (127.9.174.1:34017)
I20260807 08:35:31.378499 25044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c33f85cb90f4467ba4747ffe9e2de4e8" candidate_uuid: "0ba6cde74457404fa3dd988f7222adf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38d1b2e0ec24c2f8a607db769e540a9"
I20260807 08:35:31.378508 25242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c33f85cb90f4467ba4747ffe9e2de4e8" candidate_uuid: "0ba6cde74457404fa3dd988f7222adf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be714da5fe64440ba59c825ee5040eda"
I20260807 08:35:31.378659 25044 raft_consensus.cc:3037] T c33f85cb90f4467ba4747ffe9e2de4e8 P a38d1b2e0ec24c2f8a607db769e540a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:31.378659 25242 raft_consensus.cc:3037] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:35:31.380363 25242 raft_consensus.cc:2454] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ba6cde74457404fa3dd988f7222adf6 in term 1.
I20260807 08:35:31.380363 25044 raft_consensus.cc:2454] T c33f85cb90f4467ba4747ffe9e2de4e8 P a38d1b2e0ec24c2f8a607db769e540a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ba6cde74457404fa3dd988f7222adf6 in term 1.
I20260807 08:35:31.380666 25355 leader_election.cc:302] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ba6cde74457404fa3dd988f7222adf6, be714da5fe64440ba59c825ee5040eda; no voters:
I20260807 08:35:31.380870 25571 raft_consensus.cc:2781] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:35:31.381191 25571 raft_consensus.cc:686] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [term 1 LEADER]: Becoming Leader. State: Replica: 0ba6cde74457404fa3dd988f7222adf6, State: Running, Role: LEADER
I20260807 08:35:31.381629 25571 consensus_queue.cc:237] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be714da5fe64440ba59c825ee5040eda" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44807 } } peers { permanent_uuid: "a38d1b2e0ec24c2f8a607db769e540a9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 34017 } } peers { permanent_uuid: "0ba6cde74457404fa3dd988f7222adf6" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40199 } }
I20260807 08:35:31.385582 24815 catalog_manager.cc:5495] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ba6cde74457404fa3dd988f7222adf6 (127.9.174.3). New cstate: current_term: 1 leader_uuid: "0ba6cde74457404fa3dd988f7222adf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be714da5fe64440ba59c825ee5040eda" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38d1b2e0ec24c2f8a607db769e540a9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ba6cde74457404fa3dd988f7222adf6" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 40199 } health_report { overall_health: HEALTHY } } }
I20260807 08:35:31.410054 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:31.405121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37648 (local address 127.9.174.3:40199)
0807 08:35:31.405308 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 08:35:31.405314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:35:31.405333 (+ 19us) server_negotiation.cc:407] Connection header received
0807 08:35:31.405531 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:35:31.405534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:35:31.405610 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:35:31.405744 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:35:31.406058 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.406764 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.407938 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:35:31.408178 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:35:31.408278 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:35:31.408694 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:35:31.409345 (+ 651us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:35:31.409468 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 08:35:31.409658 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 08:35:31.409795 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 08:35:31.410077 25323 tablet.cc:2369] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:35:31.418789 25242 raft_consensus.cc:1261] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda [term 1 FOLLOWER]: Refusing update from remote peer 0ba6cde74457404fa3dd988f7222adf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:35:31.418788 25044 raft_consensus.cc:1261] T c33f85cb90f4467ba4747ffe9e2de4e8 P a38d1b2e0ec24c2f8a607db769e540a9 [term 1 FOLLOWER]: Refusing update from remote peer 0ba6cde74457404fa3dd988f7222adf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:35:31.419786 25571 consensus_queue.cc:1035] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be714da5fe64440ba59c825ee5040eda" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:35:31.420087 25573 consensus_queue.cc:1035] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38d1b2e0ec24c2f8a607db769e540a9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:35:31.431453 25579 mvcc.cc:205] Tried to move back new op lower bound from 7315831731879989248 to 7315831731742314496. Current Snapshot: MvccSnapshot[applied={T|T < 7315831731879989248}]
I20260807 08:35:31.432175 25580 mvcc.cc:205] Tried to move back new op lower bound from 7315831731879989248 to 7315831731742314496. Current Snapshot: MvccSnapshot[applied={T|T < 7315831731879989248}]
I20260807 08:35:31.446332 24815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:35:31.446571 24815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:35:31.449790 24815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5cd6c88e61043088212610f7824d1f5: Sending DeleteTablet for 3 replicas of tablet c33f85cb90f4467ba4747ffe9e2de4e8
I20260807 08:35:31.450994 25017 tablet_service.cc:1514] Processing DeleteTablet for tablet c33f85cb90f4467ba4747ffe9e2de4e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:31 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:54598
I20260807 08:35:31.450977 25211 tablet_service.cc:1514] Processing DeleteTablet for tablet c33f85cb90f4467ba4747ffe9e2de4e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:31 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:53800
I20260807 08:35:31.451148 25430 tablet_service.cc:1514] Processing DeleteTablet for tablet c33f85cb90f4467ba4747ffe9e2de4e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:35:31 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:37634
I20260807 08:35:31.452005 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 24915
I20260807 08:35:31.451989 25591 tablet_replica.cc:330] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda: stopping tablet replica
I20260807 08:35:31.452006 25590 tablet_replica.cc:330] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6: stopping tablet replica
I20260807 08:35:31.452529 25591 raft_consensus.cc:2229] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:35:31.452546 25590 raft_consensus.cc:2229] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:35:31.453049 25591 raft_consensus.cc:2258] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:31.453346 25590 raft_consensus.cc:2258] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:35:31.457549 25591 ts_tablet_manager.cc:1905] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:35:31.461505 25591 ts_tablet_manager.cc:1918] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:35:31.461632 25591 log.cc:1196] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/c33f85cb90f4467ba4747ffe9e2de4e8
I20260807 08:35:31.461938 25591 ts_tablet_manager.cc:1939] T c33f85cb90f4467ba4747ffe9e2de4e8 P be714da5fe64440ba59c825ee5040eda: Deleting consensus metadata
I20260807 08:35:31.463727 24789 catalog_manager.cc:4841] TS be714da5fe64440ba59c825ee5040eda (127.9.174.2:44807): tablet c33f85cb90f4467ba4747ffe9e2de4e8 (table test-table [id=d08ac9d44d1c49cdb41ec7aed881ca7c]) successfully deleted
I20260807 08:35:31.465261 25590 ts_tablet_manager.cc:1905] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:35:31.468506 25590 ts_tablet_manager.cc:1918] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:35:31.468643 25590 log.cc:1196] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/c33f85cb90f4467ba4747ffe9e2de4e8
I20260807 08:35:31.469020 25590 ts_tablet_manager.cc:1939] T c33f85cb90f4467ba4747ffe9e2de4e8 P 0ba6cde74457404fa3dd988f7222adf6: Deleting consensus metadata
I20260807 08:35:31.470609 24793 catalog_manager.cc:4841] TS 0ba6cde74457404fa3dd988f7222adf6 (127.9.174.3:40199): tablet c33f85cb90f4467ba4747ffe9e2de4e8 (table test-table [id=d08ac9d44d1c49cdb41ec7aed881ca7c]) successfully deleted
W20260807 08:35:31.471727 24795 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:34017 (error 108)
I20260807 08:35:31.472884 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 25124
I20260807 08:35:31.473104 25555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:35:31.472694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:34017 (local address 127.0.0.1:54608)
0807 08:35:31.472875 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 08:35:31.472965 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:34017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 08:35:31.486591 24795 catalog_manager.cc:4584] TS a38d1b2e0ec24c2f8a607db769e540a9 (127.9.174.1:34017): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c33f85cb90f4467ba4747ffe9e2de4e8: Network error: Client connection negotiation failed: client connection to 127.9.174.1:34017: connect: Connection refused (error 111)
I20260807 08:35:31.496323 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 25328
I20260807 08:35:31.513691 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 24756
2026-08-07T08:35:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2918 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:35:31 dist-test-slave-h3ft krb5kdc[25600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:35:31 dist-test-slave-h3ft krb5kdc[25600](info): set up 2 sockets
Aug 07 08:35:31 dist-test-slave-h3ft krb5kdc[25600](info): commencing operation
krb5kdc: starting...
W20260807 08:35:32.767560 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:35:32 dist-test-slave-h3ft krb5kdc[25600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:35:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:35:32Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.174.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.174.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.174.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/rangeradmin_127.9.174.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.174.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/rangeradmin_127.9.174.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.174.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.174.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.174.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/rangerlookup_127.9.174.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.174.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/rangerlookup_127.9.174.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.174.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.174.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.174.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/HTTP_127.9.174.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.174.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/HTTP_127.9.174.20@KRBTEST.COM.keytab.
I20260807 08:35:32.995535 9912 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task1t_l5C/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/postgres -l logfile start
2026-08-07 08:35:35.985 UTC [25646] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 08:35:35.985 UTC [25646] LOG: listening on IPv4 address "127.9.174.20", port 45407
2026-08-07 08:35:35.990 UTC [25646] LOG: listening on Unix socket "/tmp/.s.PGSQL.45407"
2026-08-07 08:35:36.017 UTC [25650] LOG: database system was shut down at 2026-08-07 08:35:34 UTC
2026-08-07 08:35:36.023 UTC [25646] LOG: database system is ready to accept connections
I20260807 08:35:37.183260 9912 mini_postgres.cc:96] Postgres bound to 45407
2026-08-07 08:35:37.193 UTC [25663] FATAL: database "slave" does not exist
127.9.174.20:45407 - accepting connections
I20260807 08:35:37.194415 9912 mini_ranger.cc:161] Starting Ranger...
I20260807 08:35:37.216485 9912 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 08:35:37.820899 9912 mini_ranger.cc:87] Created ranger Postgres database
I20260807 08:35:37.821028 9912 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-admin
2026-08-07 08:35:38,229 [I] DB FLAVOR :POSTGRES
2026-08-07 08:35:38,230 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 08:35:38,230 [I] Checking connection..
2026-08-07 08:35:38,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 08:35:38,532 [I] Checking connection passed.
2026-08-07 08:35:38,532 [I] --------- Verifying version history table ---------
2026-08-07 08:35:38,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:35:38,849 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 08:35:38,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:35:39,139 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 08:35:39,139 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 08:35:39,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 08:35:39.421 UTC [25762] WARNING: there is no transaction in progress
2026-08-07 08:35:39,432 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 08:35:39,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:35:39,763 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 08:35:39,764 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 08:35:39,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 08:35:40,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 08:35:40,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h3ft.c.gcp-upstream.internal','N') ;"
2026-08-07 08:35:40,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 08:35:40,987 [I] Table x_portal_user does not exist in database ranger
2026-08-07 08:35:40,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 08:35:41,304 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 08:35:41,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 08:35:41,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 08:35:41,967 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 08:35:41,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 08:35:43.174 UTC [25927] WARNING: there is no transaction in progress
2026-08-07 08:35:43.194 UTC [25927] WARNING: there is no transaction in progress
2026-08-07 08:35:43.217 UTC [25927] WARNING: there is no transaction in progress
2026-08-07 08:35:43.236 UTC [25927] WARNING: there is no transaction in progress
2026-08-07 08:35:43.257 UTC [25927] WARNING: there is no transaction in progress
2026-08-07 08:35:43.281 UTC [25927] WARNING: there is no transaction in progress
2026-08-07 08:35:43.357 UTC [25927] WARNING: there is no transaction in progress
2026-08-07 08:35:43.366 UTC [25927] WARNING: there is no transaction in progress
2026-08-07 08:35:43.384 UTC [25927] WARNING: there is no transaction in progress
2026-08-07 08:35:43.397 UTC [25927] WARNING: there is no transaction in progress
2026-08-07 08:35:43.410 UTC [25927] WARNING: there is no transaction in progress
2026-08-07 08:35:43.427 UTC [25927] WARNING: there is no transaction in progress
2026-08-07 08:35:43.441 UTC [25927] WARNING: there is no transaction in progress
2026-08-07 08:35:43.449 UTC [25927] WARNING: there is no transaction in progress
2026-08-07 08:35:43.459 UTC [25927] WARNING: there is no transaction in progress
2026-08-07 08:35:43.469 UTC [25927] WARNING: there is no transaction in progress
2026-08-07 08:35:44,326 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 08:35:44,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 08:35:44,624 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 08:35:44,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 08:35:44,924 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 08:35:44,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 08:35:45,239 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 08:35:45,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 08:35:45,522 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 08:35:45,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h3ft.c.gcp-upstream.internal';"
2026-08-07 08:35:46,119 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 08:35:46,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 08:35:46,419 [I] DB_PATCHES have already been applied
I20260807 08:35:46.426865 9912 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-admin:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task1t_l5C/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 08:35:46.426970 9912 mini_ranger.cc:193] Using host: 127.9.174.20
I20260807 08:35:46.432832 9912 mini_ranger.cc:227] Ranger admin URL: http://127.9.174.20:52315
Aug 07, 2026 8:35:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:35:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:35:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 8:35:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 8:35:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 8:35:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 8:35:47 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 08:35:47.864969 9912 mini_ranger.cc:160] Time spent starting Ranger: real 10.671s user 0.006s sys 0.010s
Aug 07, 2026 8:36:12 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 8:36:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 8:36:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 8:36:12 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 8:36:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 8:36:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 8:36:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 8:36:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 8:36:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 8:36:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 8:36:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 8:36:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 8:36:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 8:36:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 8:36:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 8:36:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 8:36:12 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 8:36:13 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 08:36:14.325181 9912 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.174.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.174.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.174.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/rangerkms_127.9.174.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.174.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/rangerkms_127.9.174.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.174.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.174.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.174.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/HTTP_127.9.174.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.174.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/HTTP_127.9.174.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 08:36:14 dist-test-slave-h3ft krb5kdc[25600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091774, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 08:36:14.464004 9912 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 08:36:14.481338 9912 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 08:36:15.565791 9912 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 08:36:15.565910 9912 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-kms
I20260807 08:36:15.566066 9912 mini_ranger_kms.cc:227] Using postgres at 127.9.174.20:45407
2026-08-07 08:36:15,855 [I] DB FLAVOR :POSTGRES
2026-08-07 08:36:15,856 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 08:36:15,856 [I] Checking connection
2026-08-07 08:36:15,856 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 08:36:16,151 [I] connection success
2026-08-07 08:36:16,151 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 08:36:16,152 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 08:36:16,152 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 08:36:16,450 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 08:36:16,450 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 08:36:16,450 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 08:36:16,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:45407/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 08:36:16,782 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 08:36:17.388334 9912 mini_ranger_kms.cc:310] Created kms service
I20260807 08:36:17.649680 9912 mini_ranger_kms.cc:326] Created kudu user
I20260807 08:36:17.716748 9912 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 08:36:17 dist-test-slave-h3ft krb5kdc[25600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091774, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
I20260807 08:36:18.182155 9912 mini_ranger_kms.cc:384] Added ranger policy
I20260807 08:36:18.182389 9912 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-kms:/tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 08:36:18.182425 9912 mini_ranger_kms.cc:240] Using host: 127.9.174.20
I20260807 08:36:18.188973 9912 mini_ranger_kms.cc:276] Ranger KMS PID: 26206
I20260807 08:36:18.189203 9912 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.174.20:46783
Aug 07, 2026 8:36:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:36:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:36:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 8:36:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 8:36:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 8:36:19 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 08:36:20 dist-test-slave-h3ft krb5kdc[25600](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786091780, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.174.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:36:21.085987 9912 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.622s user 0.025s sys 0.251s
I20260807 08:36:21.086185 9912 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 08:36:21.086335 9912 mini_ranger_kms.cc:401] 127.9.174.20:46783/kms/v1/keys
Aug 07 08:36:21 dist-test-slave-h3ft krb5kdc[25600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091780, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.174.20@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
Aug 07 08:36:21 dist-test-slave-h3ft krb5kdc[25600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091780, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.174.20@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
Aug 07 08:36:24 dist-test-slave-h3ft krb5kdc[25600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:36:24.438606 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:37649
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:38815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:37649
--ranger_config_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.174.20:46783/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:36:24.576435 26293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:36:24.576930 26293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:36:24.576977 26293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:36:24.581568 26293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:36:24.581660 26293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:36:24.581682 26293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:36:24.581699 26293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:36:24.581715 26293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:36:24.588349 26293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:38815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.174.20:46783/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:37649
--ranger_config_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:37649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:36:24.589892 26293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:36:24.591395 26293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:36:24.602793 26299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:36:24.602804 26300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:36:24.602854 26293 server_base.cc:1034] running on GCE node
W20260807 08:36:24.602797 26303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:36:24.603991 26293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:36:24.605594 26293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:36:24.606781 26293 hybrid_clock.cc:648] HybridClock initialized: now 1786091784606774 us; error 39 us; skew 500 ppm
Aug 07 08:36:24 dist-test-slave-h3ft krb5kdc[25600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091784, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:36:24.614706 26293 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:36:24.616330 26293 webserver.cc:460] Webserver started at http://127.9.174.62:37305/ using document root <none> and password file <none>
I20260807 08:36:24.617296 26293 fs_manager.cc:359] Metadata directory not provided
I20260807 08:36:24.617399 26293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:36:24.617687 26293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:36:24 dist-test-slave-h3ft krb5kdc[25600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091784, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
I20260807 08:36:24.687614 26293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "1f931d94846f4dd392de1ad304728744"
format_stamp: "Formatted at 2026-08-07 08:36:24 on dist-test-slave-h3ft"
server_key: "00e55616198c5a731a84e5d35cb16769"
server_key_iv: "9b2f2c55192d48d926d6b3b85d17eb1d"
server_key_version: "kuduclusterkey@0"
I20260807 08:36:24.688489 26293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "1f931d94846f4dd392de1ad304728744"
format_stamp: "Formatted at 2026-08-07 08:36:24 on dist-test-slave-h3ft"
server_key: "00e55616198c5a731a84e5d35cb16769"
server_key_iv: "9b2f2c55192d48d926d6b3b85d17eb1d"
server_key_version: "kuduclusterkey@0"
I20260807 08:36:24.694285 26293 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:36:24.713385 26314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:36:24.715273 26293 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:36:24.715461 26293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "1f931d94846f4dd392de1ad304728744"
format_stamp: "Formatted at 2026-08-07 08:36:24 on dist-test-slave-h3ft"
server_key: "00e55616198c5a731a84e5d35cb16769"
server_key_iv: "9b2f2c55192d48d926d6b3b85d17eb1d"
server_key_version: "kuduclusterkey@0"
I20260807 08:36:24.715651 26293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:36:24.728479 26293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:36:24.732198 26293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:36:24.732474 26293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:36:24.741881 26293 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:37649
I20260807 08:36:24.741896 26392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:37649 every 8 connection(s)
I20260807 08:36:24.743229 26293 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:36:24.747197 26393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:36:24.750458 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 26293
I20260807 08:36:24.750629 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:36:24.751039 9912 external_mini_cluster.cc:1428] Setting key 2acf7c3c33a6705930aecff9769b4d43
I20260807 08:36:24.755558 26393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744: Bootstrap starting.
I20260807 08:36:24.758333 26393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744: Neither blocks nor log segments found. Creating new log.
I20260807 08:36:24.759325 26393 log.cc:824] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:36:24 dist-test-slave-h3ft krb5kdc[25600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:36:24.762306 26393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744: No bootstrap required, opened a new log
I20260807 08:36:24.767196 26393 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f931d94846f4dd392de1ad304728744" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37649 } }
I20260807 08:36:24.767480 26393 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:36:24.767544 26393 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f931d94846f4dd392de1ad304728744, State: Initialized, Role: FOLLOWER
I20260807 08:36:24.768103 26393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f931d94846f4dd392de1ad304728744" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37649 } }
I20260807 08:36:24.768242 26393 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:36:24.768321 26397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:24.753044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53960 (local address 127.9.174.62:37649)
0807 08:36:24.753661 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0807 08:36:24.753679 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:36:24.753726 (+ 47us) server_negotiation.cc:407] Connection header received
0807 08:36:24.754481 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:36:24.754501 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:36:24.754902 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:36:24.755388 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:36:24.756364 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:24.757488 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:24.758441 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:24.759093 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:24.759278 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:24.762590 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:36:24.762597 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:36:24.762615 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:36:24.762648 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:36:24.764644 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:36:24.765219 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:36:24.765225 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:36:24.765230 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:36:24.765391 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:36:24.765727 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:36:24.765730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:36:24.765731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:36:24.766288 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:36:24.766509 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 08:36:24.766831 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 08:36:24.767228 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":192,"threads_started":1}
I20260807 08:36:24.768342 26393 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:36:24.768527 26393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:36:24.769632 26393 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f931d94846f4dd392de1ad304728744" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37649 } }
I20260807 08:36:24.770033 26393 leader_election.cc:302] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f931d94846f4dd392de1ad304728744; no voters:
I20260807 08:36:24.770402 26393 leader_election.cc:288] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:36:24.770539 26400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:36:24.770794 26400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [term 1 LEADER]: Becoming Leader. State: Replica: 1f931d94846f4dd392de1ad304728744, State: Running, Role: LEADER
I20260807 08:36:24.771502 26400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f931d94846f4dd392de1ad304728744" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37649 } }
I20260807 08:36:24.771982 26393 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:36:24.772832 26400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f931d94846f4dd392de1ad304728744. Latest consensus state: current_term: 1 leader_uuid: "1f931d94846f4dd392de1ad304728744" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f931d94846f4dd392de1ad304728744" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37649 } } }
I20260807 08:36:24.773095 26400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [sys.catalog]: This master's current role is: LEADER
I20260807 08:36:24.773380 26401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f931d94846f4dd392de1ad304728744" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f931d94846f4dd392de1ad304728744" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 37649 } } }
I20260807 08:36:24.773564 26401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744 [sys.catalog]: This master's current role is: LEADER
I20260807 08:36:24.773663 26415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:36:24.777921 26415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:36:24.778479 26393 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 08:36:24.790019 26415 catalog_manager.cc:1347] Generated new cluster ID: 014589d9feda4d7e98eef7c295b69886
I20260807 08:36:24.790145 26415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:36:24.810918 26415 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:36:24.812565 26415 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:36:24.822520 26415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744: Generated new TSK 0
I20260807 08:36:24.823619 26415 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:36:25.616052 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.174.20:46783/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37649
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:38815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:36:25.789929 26466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:36:25.790438 26466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:36:25.790512 26466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:36:25.797238 26466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:36:25.797447 26466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:36:25.797773 26466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:36:25.805768 26466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:38815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.174.20:46783/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:36:25.807427 26466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:36:25.808779 26466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:36:25.823498 26478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:36:25.823506 26476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:36:25.823724 26466 server_base.cc:1034] running on GCE node
W20260807 08:36:25.823506 26480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:36:25.824551 26466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:36:25.825578 26466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:36:25.828930 26466 hybrid_clock.cc:648] HybridClock initialized: now 1786091785828900 us; error 46 us; skew 500 ppm
Aug 07 08:36:25 dist-test-slave-h3ft krb5kdc[25600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091785, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:36:25.832875 26466 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:36:25.834296 26466 webserver.cc:460] Webserver started at http://127.9.174.1:37381/ using document root <none> and password file <none>
I20260807 08:36:25.835170 26466 fs_manager.cc:359] Metadata directory not provided
I20260807 08:36:25.835278 26466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:36:25.835578 26466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:36:25 dist-test-slave-h3ft krb5kdc[25600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091785, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
I20260807 08:36:25.858187 26466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "81744ac4b825447a9ecf3dc9696148fe"
format_stamp: "Formatted at 2026-08-07 08:36:25 on dist-test-slave-h3ft"
server_key: "72d69f96132cdc0bc70bedb6f7805ae9"
server_key_iv: "c7ebb5af7510e0e41e0e288a6996fe2f"
server_key_version: "kuduclusterkey@0"
I20260807 08:36:25.859217 26466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "81744ac4b825447a9ecf3dc9696148fe"
format_stamp: "Formatted at 2026-08-07 08:36:25 on dist-test-slave-h3ft"
server_key: "72d69f96132cdc0bc70bedb6f7805ae9"
server_key_iv: "c7ebb5af7510e0e41e0e288a6996fe2f"
server_key_version: "kuduclusterkey@0"
I20260807 08:36:25.869568 26466 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.007s sys 0.000s
I20260807 08:36:25.903669 26492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:36:25.906302 26466 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.002s sys 0.000s
I20260807 08:36:25.906519 26466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "81744ac4b825447a9ecf3dc9696148fe"
format_stamp: "Formatted at 2026-08-07 08:36:25 on dist-test-slave-h3ft"
server_key: "72d69f96132cdc0bc70bedb6f7805ae9"
server_key_iv: "c7ebb5af7510e0e41e0e288a6996fe2f"
server_key_version: "kuduclusterkey@0"
I20260807 08:36:25.906723 26466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:36:25.934573 26466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:36:25.939622 26466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:36:25.939986 26466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:36:25.941040 26466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:36:25.942482 26466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:36:25.942605 26466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:36:25.942781 26466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:36:25.942895 26466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:36:25.957494 26466 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:44921
I20260807 08:36:25.957719 26686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:44921 every 8 connection(s)
I20260807 08:36:25.958938 26466 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:36:25.960417 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 26466
I20260807 08:36:25.960598 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:36:25.961014 9912 external_mini_cluster.cc:1428] Setting key 58fcb5bc3906f621ed21c79cddaa70c3
Aug 07 08:36:25 dist-test-slave-h3ft krb5kdc[25600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091785, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
I20260807 08:36:25.984441 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:25.961711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:48577 (local address 127.9.174.62:37649)
0807 08:36:25.962120 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0807 08:36:25.962125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:36:25.966819 (+ 4694us) server_negotiation.cc:407] Connection header received
0807 08:36:25.968641 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:36:25.968645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:36:25.968723 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:36:25.968823 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:36:25.970909 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:25.971895 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:25.974111 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:25.974506 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:25.974606 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:25.978821 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:36:25.978827 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:36:25.978832 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:36:25.978877 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:36:25.981293 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:36:25.982058 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:36:25.982062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:36:25.982064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:36:25.982170 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:36:25.982651 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:36:25.982655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:36:25.982657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:36:25.982969 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:36:25.983087 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 08:36:25.984023 (+ 936us) server_negotiation.cc:299] Negotiation successful
0807 08:36:25.984195 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":151,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
I20260807 08:36:25.987424 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:25.962218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37649 (local address 127.9.174.1:48577)
0807 08:36:25.966538 (+ 4320us) negotiation.cc:107] Waiting for socket to connect
0807 08:36:25.966627 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0807 08:36:25.968190 (+ 1563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:36:25.969255 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:36:25.969268 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:36:25.969764 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:36:25.970669 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:36:25.970693 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:36:25.972103 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:36:25.972107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:36:25.973884 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:36:25.973912 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:36:25.974922 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:36:25.974927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:36:25.975201 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:25.976280 (+ 1079us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:36:25.976312 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:36:25.978545 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:36:25.981460 (+ 2915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:36:25.981468 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:36:25.981494 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:36:25.981924 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:36:25.982288 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:36:25.982291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:36:25.982293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:36:25.982520 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:36:25.983189 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:36:25.983198 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:36:25.983652 (+ 454us) client_negotiation.cc:769] Sending connection context
0807 08:36:25.985000 (+ 1348us) client_negotiation.cc:240] Negotiation successful
0807 08:36:25.985476 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"thread_start_us":165,"threads_started":1}
I20260807 08:36:25.990145 26687 heartbeater.cc:347] Connected to a master server at 127.9.174.62:37649
I20260807 08:36:25.990504 26687 heartbeater.cc:464] Registering TS with master...
I20260807 08:36:25.991403 26687 heartbeater.cc:511] Master 127.9.174.62:37649 requested a full tablet report, sending...
I20260807 08:36:25.994907 26334 ts_manager.cc:194] Registered new tserver with Master: 81744ac4b825447a9ecf3dc9696148fe (127.9.174.1:44921)
I20260807 08:36:25.996996 26334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:48577
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:36:26.011448 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.174.20:46783/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37649
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:38815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:36:26 dist-test-slave-h3ft krb5kdc[25600](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786091786, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 08:36:26.204723 26696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:36:26.205336 26696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:36:26.205493 26696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:36:26.212318 26696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:36:26.212540 26696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:36:26.212762 26696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:36:26.221041 26696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:38815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.174.20:46783/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:36:26.222683 26696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:36:26.224223 26696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:36:26.237686 26713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:36:26.237845 26712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:36:26.237948 26717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:36:26.239602 26696 server_base.cc:1034] running on GCE node
I20260807 08:36:26.240271 26696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:36:26.241287 26696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:36:26.242550 26696 hybrid_clock.cc:648] HybridClock initialized: now 1786091786242509 us; error 57 us; skew 500 ppm
Aug 07 08:36:26 dist-test-slave-h3ft krb5kdc[25600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091786, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:36:26.246794 26696 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:36:26.248405 26696 webserver.cc:460] Webserver started at http://127.9.174.2:43783/ using document root <none> and password file <none>
I20260807 08:36:26.249243 26696 fs_manager.cc:359] Metadata directory not provided
I20260807 08:36:26.249375 26696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:36:26.249696 26696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:36:26 dist-test-slave-h3ft krb5kdc[25600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091786, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
I20260807 08:36:26.271054 26696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "18365e75788745b7b74fb5fca9fbbd60"
format_stamp: "Formatted at 2026-08-07 08:36:26 on dist-test-slave-h3ft"
server_key: "ac55eac80d9dd8d551a41a77a2ee380e"
server_key_iv: "0df47669c9a6a35e562b15ca8dd7f624"
server_key_version: "kuduclusterkey@0"
I20260807 08:36:26.273350 26696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "18365e75788745b7b74fb5fca9fbbd60"
format_stamp: "Formatted at 2026-08-07 08:36:26 on dist-test-slave-h3ft"
server_key: "ac55eac80d9dd8d551a41a77a2ee380e"
server_key_iv: "0df47669c9a6a35e562b15ca8dd7f624"
server_key_version: "kuduclusterkey@0"
I20260807 08:36:26.288704 26696 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.001s sys 0.004s
I20260807 08:36:26.318621 26728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:36:26.320923 26696 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.001s
I20260807 08:36:26.321122 26696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "18365e75788745b7b74fb5fca9fbbd60"
format_stamp: "Formatted at 2026-08-07 08:36:26 on dist-test-slave-h3ft"
server_key: "ac55eac80d9dd8d551a41a77a2ee380e"
server_key_iv: "0df47669c9a6a35e562b15ca8dd7f624"
server_key_version: "kuduclusterkey@0"
I20260807 08:36:26.321337 26696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:36:26.354038 26696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:36:26.358671 26696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:36:26.358982 26696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:36:26.360060 26696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:36:26.361413 26696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:36:26.361501 26696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:36:26.361598 26696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:36:26.361652 26696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:36:26.379240 26696 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:43363
I20260807 08:36:26.381053 26696 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:36:26.387153 26899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:43363 every 8 connection(s)
I20260807 08:36:26.388724 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 26696
I20260807 08:36:26.388940 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:36:26.389397 9912 external_mini_cluster.cc:1428] Setting key 867fc0e227b7f2ff7b8e305d88c41224
Aug 07 08:36:26 dist-test-slave-h3ft krb5kdc[25600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091786, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:36:26.405213 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:26.384043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:55993 (local address 127.9.174.62:37649)
0807 08:36:26.384259 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 08:36:26.384264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:36:26.386522 (+ 2258us) server_negotiation.cc:407] Connection header received
0807 08:36:26.388243 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:36:26.388249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:36:26.388352 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:36:26.388437 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:36:26.390754 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.391743 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.393777 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.394154 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.394251 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:26.398880 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:36:26.398887 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:36:26.398891 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:36:26.398940 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:36:26.401247 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:36:26.402272 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:36:26.402278 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:36:26.402280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:36:26.402415 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:36:26.403084 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:36:26.403089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:36:26.403091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:36:26.403439 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:36:26.403552 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 08:36:26.404678 (+ 1126us) server_negotiation.cc:299] Negotiation successful
0807 08:36:26.404920 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 08:36:26.408182 26903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:26.384681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37649 (local address 127.9.174.2:55993)
0807 08:36:26.386235 (+ 1554us) negotiation.cc:107] Waiting for socket to connect
0807 08:36:26.386354 (+ 119us) client_negotiation.cc:174] Beginning negotiation
0807 08:36:26.387893 (+ 1539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:36:26.389056 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:36:26.389069 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:36:26.389559 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:36:26.390478 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:36:26.390503 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.391965 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.391969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:36:26.393587 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:36:26.393601 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.394539 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.394544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:36:26.394801 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:26.396492 (+ 1691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:36:26.396520 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:36:26.398546 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:36:26.401486 (+ 2940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:36:26.401497 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:36:26.401528 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:36:26.402069 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:36:26.402579 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:36:26.402583 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:36:26.402585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:36:26.402879 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:36:26.403677 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:36:26.403688 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:36:26.404248 (+ 560us) client_negotiation.cc:769] Sending connection context
0807 08:36:26.405978 (+ 1730us) client_negotiation.cc:240] Negotiation successful
0807 08:36:26.406568 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1267,"thread_start_us":205,"threads_started":1}
I20260807 08:36:26.410552 26900 heartbeater.cc:347] Connected to a master server at 127.9.174.62:37649
I20260807 08:36:26.411057 26900 heartbeater.cc:464] Registering TS with master...
I20260807 08:36:26.412019 26900 heartbeater.cc:511] Master 127.9.174.62:37649 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
I20260807 08:36:26.414108 26334 ts_manager.cc:194] Registered new tserver with Master: 18365e75788745b7b74fb5fca9fbbd60 (127.9.174.2:43363)
I20260807 08:36:26.415161 26334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:55993
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:36:26.439994 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.174.20:46783/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37649
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:38815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:36:26 dist-test-slave-h3ft krb5kdc[25600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091786, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
W20260807 08:36:26.638855 26907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:36:26.639571 26907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:36:26.640479 26907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:36:26.647608 26907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:36:26.647810 26907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:36:26.647972 26907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:36:26.657239 26907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:38815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.174.20:46783/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:37649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:36:26.659528 26907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:36:26.661299 26907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:36:26.677922 26916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:36:26.678817 26915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:36:26.679622 26918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:36:26.680367 26907 server_base.cc:1034] running on GCE node
I20260807 08:36:26.681144 26907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:36:26.682186 26907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:36:26.685482 26907 hybrid_clock.cc:648] HybridClock initialized: now 1786091786685385 us; error 107 us; skew 500 ppm
Aug 07 08:36:26 dist-test-slave-h3ft krb5kdc[25600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091786, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:36:26.690708 26907 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:36:26.692420 26907 webserver.cc:460] Webserver started at http://127.9.174.3:42445/ using document root <none> and password file <none>
I20260807 08:36:26.693471 26907 fs_manager.cc:359] Metadata directory not provided
I20260807 08:36:26.693661 26907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:36:26.694010 26907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:36:26 dist-test-slave-h3ft krb5kdc[25600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091786, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
I20260807 08:36:26.717475 26907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "5eb9d3a689dd469e843aae83c9959deb"
format_stamp: "Formatted at 2026-08-07 08:36:26 on dist-test-slave-h3ft"
server_key: "aee5b19a5482d2866235f271cb996208"
server_key_iv: "fd6266ee7b0f52fdae5414d996547b1b"
server_key_version: "kuduclusterkey@0"
I20260807 08:36:26.718573 26907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "5eb9d3a689dd469e843aae83c9959deb"
format_stamp: "Formatted at 2026-08-07 08:36:26 on dist-test-slave-h3ft"
server_key: "aee5b19a5482d2866235f271cb996208"
server_key_iv: "fd6266ee7b0f52fdae5414d996547b1b"
server_key_version: "kuduclusterkey@0"
I20260807 08:36:26.728824 26907 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.002s sys 0.003s
I20260807 08:36:26.772644 26930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:36:26.777397 26907 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.002s sys 0.000s
I20260807 08:36:26.777623 26907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "5eb9d3a689dd469e843aae83c9959deb"
format_stamp: "Formatted at 2026-08-07 08:36:26 on dist-test-slave-h3ft"
server_key: "aee5b19a5482d2866235f271cb996208"
server_key_iv: "fd6266ee7b0f52fdae5414d996547b1b"
server_key_version: "kuduclusterkey@0"
I20260807 08:36:26.777822 26907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:36:26.849987 26907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:36:26.864753 26907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:36:26.865132 26907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:36:26.866082 26907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:36:26.867482 26907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:36:26.867599 26907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 08:36:26.867710 26907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:36:26.867776 26907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:36:26.893009 26907 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:45967
I20260807 08:36:26.894757 26907 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:36:26.897099 27091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:45967 every 8 connection(s)
I20260807 08:36:26.903591 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 26907
I20260807 08:36:26.903792 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:36:26.904203 9912 external_mini_cluster.cc:1428] Setting key 84cf9bb07ea8f8ac481fd85be1b34822
Aug 07 08:36:26 dist-test-slave-h3ft krb5kdc[25600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091786, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:36:26.918563 27094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:26.900689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:37649 (local address 127.9.174.3:34535)
0807 08:36:26.902927 (+ 2238us) negotiation.cc:107] Waiting for socket to connect
0807 08:36:26.903019 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0807 08:36:26.904225 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:36:26.905150 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:36:26.905162 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:36:26.905645 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:36:26.906471 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:36:26.906493 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.907581 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.907584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:36:26.908592 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:36:26.908602 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.909193 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.909198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:36:26.909407 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:26.910476 (+ 1069us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:36:26.910496 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:36:26.912318 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:36:26.914510 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:36:26.914519 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:36:26.914549 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:36:26.914879 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:36:26.915254 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:36:26.915257 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:36:26.915259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:36:26.915448 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:36:26.915942 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:36:26.915952 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:36:26.916362 (+ 410us) client_negotiation.cc:769] Sending connection context
0807 08:36:26.916621 (+ 259us) client_negotiation.cc:240] Negotiation successful
0807 08:36:26.917044 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1947,"thread_start_us":230,"threads_started":1}
I20260807 08:36:26.919129 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:26.902198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:34535 (local address 127.9.174.62:37649)
0807 08:36:26.902371 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 08:36:26.902377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:36:26.903171 (+ 794us) server_negotiation.cc:407] Connection header received
0807 08:36:26.904558 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:36:26.904562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:36:26.904648 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:36:26.904745 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:36:26.906662 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.907423 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.908737 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.908989 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.909123 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:26.912552 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:36:26.912556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:36:26.912558 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:36:26.912599 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:36:26.914377 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:36:26.915039 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:36:26.915050 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:36:26.915052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:36:26.915146 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:36:26.915572 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:36:26.915576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:36:26.915578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:36:26.915813 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:36:26.915931 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 08:36:26.916664 (+ 733us) server_negotiation.cc:299] Negotiation successful
0807 08:36:26.916820 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 08:36:26.920688 27092 heartbeater.cc:347] Connected to a master server at 127.9.174.62:37649
I20260807 08:36:26.921010 27092 heartbeater.cc:464] Registering TS with master...
I20260807 08:36:26.921801 27092 heartbeater.cc:511] Master 127.9.174.62:37649 requested a full tablet report, sending...
I20260807 08:36:26.923120 26334 ts_manager.cc:194] Registered new tserver with Master: 5eb9d3a689dd469e843aae83c9959deb (127.9.174.3:45967)
I20260807 08:36:26.923825 26334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:34535
I20260807 08:36:26.931293 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:36:26.945070 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:26.934245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53974 (local address 127.9.174.62:37649)
0807 08:36:26.934423 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 08:36:26.934428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:36:26.934744 (+ 316us) server_negotiation.cc:407] Connection header received
0807 08:36:26.934946 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:36:26.934949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:36:26.935095 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:36:26.935210 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:36:26.936076 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.936729 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.937706 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.937985 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.938112 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:26.940088 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:36:26.940093 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:36:26.940102 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:36:26.940193 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:36:26.941983 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:36:26.942484 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:36:26.942490 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:36:26.942492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:36:26.942594 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:36:26.942921 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:36:26.942925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:36:26.942928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:36:26.943277 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:36:26.943467 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 08:36:26.944676 (+ 1209us) server_negotiation.cc:299] Negotiation successful
0807 08:36:26.944844 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 08:36:26.950969 26334 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:36:26.957098 26334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:36:26.986428 27109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:26.972474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:43363 (local address 127.0.0.1:49002)
0807 08:36:26.974023 (+ 1549us) negotiation.cc:107] Waiting for socket to connect
0807 08:36:26.974100 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 08:36:26.974271 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:36:26.975830 (+ 1559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:36:26.975838 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:36:26.975879 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:36:26.976064 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:36:26.976072 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.980993 (+ 4921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.980997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:36:26.982442 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:36:26.982457 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.985476 (+ 3019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.985481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:36:26.985716 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:26.985806 (+ 90us) client_negotiation.cc:769] Sending connection context
0807 08:36:26.985951 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 08:36:26.986175 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1336,"mutex_wait_us":102,"thread_start_us":184,"threads_started":1}
I20260807 08:36:26.987391 27114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:26.972745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49002 (local address 127.9.174.2:43363)
0807 08:36:26.974977 (+ 2232us) server_negotiation.cc:206] Beginning negotiation
0807 08:36:26.974986 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:36:26.975049 (+ 63us) server_negotiation.cc:407] Connection header received
0807 08:36:26.975153 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:36:26.975161 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:36:26.975431 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:36:26.975610 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:36:26.979073 (+ 3463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.980395 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.983600 (+ 3205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.985296 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.986716 (+ 1420us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:26.986968 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 08:36:26.987071 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 08:36:26.987204 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2069,"thread_start_us":131,"threads_started":1}
I20260807 08:36:26.989465 27115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:26.973887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:44921 (local address 127.0.0.1:39114)
0807 08:36:26.976033 (+ 2146us) negotiation.cc:107] Waiting for socket to connect
0807 08:36:26.976052 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:36:26.976178 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:36:26.977030 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:36:26.977034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:36:26.977057 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:36:26.977211 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:36:26.977221 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.980289 (+ 3068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.980294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:36:26.984850 (+ 4556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:36:26.984879 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.988749 (+ 3870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.988755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:36:26.988963 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:26.989031 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 08:36:26.989142 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 08:36:26.989250 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2071,"mutex_wait_us":1,"thread_start_us":122,"threads_started":1}
I20260807 08:36:26.990597 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:26.972348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:45967 (local address 127.0.0.1:41234)
0807 08:36:26.974366 (+ 2018us) negotiation.cc:107] Waiting for socket to connect
0807 08:36:26.974381 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:36:26.974540 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:36:26.977143 (+ 2603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:36:26.977147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:36:26.977167 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:36:26.977286 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:36:26.977293 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.980588 (+ 3295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.980592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:36:26.983645 (+ 3053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:36:26.983660 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.990079 (+ 6419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.990085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:36:26.990211 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:26.990242 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 08:36:26.990312 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 08:36:26.990423 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1000,"mutex_wait_us":62,"thread_start_us":144,"threads_started":1}
I20260807 08:36:26.991725 27113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:26.972541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.9.174.3:45967)
0807 08:36:26.976427 (+ 3886us) server_negotiation.cc:206] Beginning negotiation
0807 08:36:26.976442 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:36:26.976469 (+ 27us) server_negotiation.cc:407] Connection header received
0807 08:36:26.976559 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:36:26.976565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:36:26.976753 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:36:26.976950 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:36:26.977441 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.978850 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.983883 (+ 5033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.985251 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.990981 (+ 5730us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:26.991272 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 08:36:26.991367 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 08:36:26.991519 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3746,"thread_start_us":217,"threads_started":1}
I20260807 08:36:26.994771 26997 tablet_service.cc:1467] Processing CreateTablet for tablet d04dd94897dc42a5ae93c67e1cf47d4a (DEFAULT_TABLE table=test-table [id=25940e6ceb7746378bddb4fd97ebc1c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:36:26.994593 26792 tablet_service.cc:1467] Processing CreateTablet for tablet d04dd94897dc42a5ae93c67e1cf47d4a (DEFAULT_TABLE table=test-table [id=25940e6ceb7746378bddb4fd97ebc1c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:36:26.996232 26792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d04dd94897dc42a5ae93c67e1cf47d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:36:26.996230 26997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d04dd94897dc42a5ae93c67e1cf47d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:36:27.001220 26687 heartbeater.cc:503] Master 127.9.174.62:37649 was elected leader, sending a full tablet report...
I20260807 08:36:27.007772 27116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:26.974855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39114 (local address 127.9.174.1:44921)
0807 08:36:26.975845 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0807 08:36:26.975853 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:36:26.976302 (+ 449us) server_negotiation.cc:407] Connection header received
0807 08:36:26.976423 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:36:26.976430 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:36:26.976648 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:36:26.976818 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:36:26.977426 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.980094 (+ 2668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.985465 (+ 5371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:26.988365 (+ 2900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:26.988519 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:27.007210 (+ 18691us) server_negotiation.cc:1035] Waiting for connection context
0807 08:36:27.007313 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 08:36:27.007520 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":853,"thread_start_us":193,"threads_started":1}
I20260807 08:36:27.011116 27117 tablet_bootstrap.cc:492] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60: Bootstrap starting.
I20260807 08:36:27.011992 26582 tablet_service.cc:1467] Processing CreateTablet for tablet d04dd94897dc42a5ae93c67e1cf47d4a (DEFAULT_TABLE table=test-table [id=25940e6ceb7746378bddb4fd97ebc1c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:36:27.013754 26582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d04dd94897dc42a5ae93c67e1cf47d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:36:27.014642 27117 tablet_bootstrap.cc:654] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60: Neither blocks nor log segments found. Creating new log.
I20260807 08:36:27.017690 27117 log.cc:824] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60: Log is configured to *not* fsync() on all Append() calls
I20260807 08:36:27.020967 27117 tablet_bootstrap.cc:492] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60: No bootstrap required, opened a new log
I20260807 08:36:27.021256 27117 ts_tablet_manager.cc:1397] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 08:36:27.026676 27125 tablet_bootstrap.cc:492] T d04dd94897dc42a5ae93c67e1cf47d4a P 81744ac4b825447a9ecf3dc9696148fe: Bootstrap starting.
I20260807 08:36:27.029510 27117 raft_consensus.cc:348] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb9d3a689dd469e843aae83c9959deb" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45967 } } peers { permanent_uuid: "18365e75788745b7b74fb5fca9fbbd60" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43363 } } peers { permanent_uuid: "81744ac4b825447a9ecf3dc9696148fe" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 44921 } }
I20260807 08:36:27.029918 27117 raft_consensus.cc:374] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:36:27.029994 27117 raft_consensus.cc:729] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18365e75788745b7b74fb5fca9fbbd60, State: Initialized, Role: FOLLOWER
I20260807 08:36:27.030663 27125 tablet_bootstrap.cc:654] T d04dd94897dc42a5ae93c67e1cf47d4a P 81744ac4b825447a9ecf3dc9696148fe: Neither blocks nor log segments found. Creating new log.
I20260807 08:36:27.030740 27117 consensus_queue.cc:260] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb9d3a689dd469e843aae83c9959deb" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45967 } } peers { permanent_uuid: "18365e75788745b7b74fb5fca9fbbd60" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43363 } } peers { permanent_uuid: "81744ac4b825447a9ecf3dc9696148fe" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 44921 } }
I20260807 08:36:27.032138 27125 log.cc:824] T d04dd94897dc42a5ae93c67e1cf47d4a P 81744ac4b825447a9ecf3dc9696148fe: Log is configured to *not* fsync() on all Append() calls
I20260807 08:36:27.032328 27117 ts_tablet_manager.cc:1428] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260807 08:36:27.032933 26900 heartbeater.cc:503] Master 127.9.174.62:37649 was elected leader, sending a full tablet report...
I20260807 08:36:27.033649 27119 tablet_bootstrap.cc:492] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb: Bootstrap starting.
I20260807 08:36:27.037686 27125 tablet_bootstrap.cc:492] T d04dd94897dc42a5ae93c67e1cf47d4a P 81744ac4b825447a9ecf3dc9696148fe: No bootstrap required, opened a new log
I20260807 08:36:27.037979 27125 ts_tablet_manager.cc:1397] T d04dd94897dc42a5ae93c67e1cf47d4a P 81744ac4b825447a9ecf3dc9696148fe: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260807 08:36:27.045221 27125 raft_consensus.cc:348] T d04dd94897dc42a5ae93c67e1cf47d4a P 81744ac4b825447a9ecf3dc9696148fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb9d3a689dd469e843aae83c9959deb" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45967 } } peers { permanent_uuid: "18365e75788745b7b74fb5fca9fbbd60" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43363 } } peers { permanent_uuid: "81744ac4b825447a9ecf3dc9696148fe" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 44921 } }
I20260807 08:36:27.045869 27125 raft_consensus.cc:374] T d04dd94897dc42a5ae93c67e1cf47d4a P 81744ac4b825447a9ecf3dc9696148fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:36:27.046007 27125 raft_consensus.cc:729] T d04dd94897dc42a5ae93c67e1cf47d4a P 81744ac4b825447a9ecf3dc9696148fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81744ac4b825447a9ecf3dc9696148fe, State: Initialized, Role: FOLLOWER
I20260807 08:36:27.046535 27119 tablet_bootstrap.cc:654] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb: Neither blocks nor log segments found. Creating new log.
I20260807 08:36:27.046972 27125 consensus_queue.cc:260] T d04dd94897dc42a5ae93c67e1cf47d4a P 81744ac4b825447a9ecf3dc9696148fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb9d3a689dd469e843aae83c9959deb" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45967 } } peers { permanent_uuid: "18365e75788745b7b74fb5fca9fbbd60" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43363 } } peers { permanent_uuid: "81744ac4b825447a9ecf3dc9696148fe" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 44921 } }
I20260807 08:36:27.047823 27119 log.cc:824] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb: Log is configured to *not* fsync() on all Append() calls
I20260807 08:36:27.048738 27125 ts_tablet_manager.cc:1428] T d04dd94897dc42a5ae93c67e1cf47d4a P 81744ac4b825447a9ecf3dc9696148fe: Time spent starting tablet: real 0.011s user 0.006s sys 0.003s
I20260807 08:36:27.051120 27119 tablet_bootstrap.cc:492] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb: No bootstrap required, opened a new log
I20260807 08:36:27.051419 27119 ts_tablet_manager.cc:1397] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.004s
I20260807 08:36:27.057293 27119 raft_consensus.cc:348] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb9d3a689dd469e843aae83c9959deb" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45967 } } peers { permanent_uuid: "18365e75788745b7b74fb5fca9fbbd60" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43363 } } peers { permanent_uuid: "81744ac4b825447a9ecf3dc9696148fe" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 44921 } }
I20260807 08:36:27.057736 27119 raft_consensus.cc:374] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:36:27.057816 27119 raft_consensus.cc:729] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eb9d3a689dd469e843aae83c9959deb, State: Initialized, Role: FOLLOWER
I20260807 08:36:27.058537 27119 consensus_queue.cc:260] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb9d3a689dd469e843aae83c9959deb" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45967 } } peers { permanent_uuid: "18365e75788745b7b74fb5fca9fbbd60" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43363 } } peers { permanent_uuid: "81744ac4b825447a9ecf3dc9696148fe" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 44921 } }
I20260807 08:36:27.060101 27119 ts_tablet_manager.cc:1428] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb: Time spent starting tablet: real 0.009s user 0.007s sys 0.001s
I20260807 08:36:27.060616 27092 heartbeater.cc:503] Master 127.9.174.62:37649 was elected leader, sending a full tablet report...
Aug 07 08:36:27 dist-test-slave-h3ft krb5kdc[25600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091786, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
W20260807 08:36:27.133692 26902 tablet.cc:2369] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:36:27.148411 27093 tablet.cc:2369] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:36:27.210356 27136 raft_consensus.cc:482] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:36:27.210749 27136 raft_consensus.cc:504] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb9d3a689dd469e843aae83c9959deb" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45967 } } peers { permanent_uuid: "18365e75788745b7b74fb5fca9fbbd60" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43363 } } peers { permanent_uuid: "81744ac4b825447a9ecf3dc9696148fe" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 44921 } }
W20260807 08:36:27.212705 26690 tablet.cc:2369] T d04dd94897dc42a5ae93c67e1cf47d4a P 81744ac4b825447a9ecf3dc9696148fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:36:27.212836 27136 leader_election.cc:288] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18365e75788745b7b74fb5fca9fbbd60 (127.9.174.2:43363), 81744ac4b825447a9ecf3dc9696148fe (127.9.174.1:44921)
I20260807 08:36:27.220826 27094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:27.213399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:43363 (local address 127.9.174.3:59715)
0807 08:36:27.213655 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 08:36:27.213691 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:36:27.213812 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:36:27.216003 (+ 2191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:36:27.216007 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:36:27.216034 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:36:27.216180 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:36:27.216187 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:36:27.217356 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:36:27.217361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:36:27.218737 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:36:27.218749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:36:27.220298 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:36:27.220302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:36:27.220424 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:27.220453 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 08:36:27.220527 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 08:36:27.220614 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260807 08:36:27.221838 27114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:27.214628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:59715 (local address 127.9.174.2:43363)
0807 08:36:27.214804 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 08:36:27.214809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:36:27.214827 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:36:27.214904 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:36:27.214907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:36:27.214960 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:36:27.215802 (+ 842us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:36:27.216323 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:27.217202 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:27.218910 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:27.219634 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:27.219737 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:27.219820 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:36:27.221586 (+ 1766us) server_negotiation.cc:299] Negotiation successful
0807 08:36:27.221696 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 08:36:27.222661 26822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d04dd94897dc42a5ae93c67e1cf47d4a" candidate_uuid: "5eb9d3a689dd469e843aae83c9959deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18365e75788745b7b74fb5fca9fbbd60" is_pre_election: true
I20260807 08:36:27.223171 26822 raft_consensus.cc:2454] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5eb9d3a689dd469e843aae83c9959deb in term 0.
I20260807 08:36:27.223881 26936 leader_election.cc:302] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18365e75788745b7b74fb5fca9fbbd60, 5eb9d3a689dd469e843aae83c9959deb; no voters:
I20260807 08:36:27.224292 27136 raft_consensus.cc:2781] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:36:27.224417 27136 raft_consensus.cc:482] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:36:27.224473 27136 raft_consensus.cc:3037] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:36:27.226861 27136 raft_consensus.cc:504] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb9d3a689dd469e843aae83c9959deb" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45967 } } peers { permanent_uuid: "18365e75788745b7b74fb5fca9fbbd60" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43363 } } peers { permanent_uuid: "81744ac4b825447a9ecf3dc9696148fe" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 44921 } }
I20260807 08:36:27.227761 27136 leader_election.cc:288] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [CANDIDATE]: Term 1 election: Requested vote from peers 18365e75788745b7b74fb5fca9fbbd60 (127.9.174.2:43363), 81744ac4b825447a9ecf3dc9696148fe (127.9.174.1:44921)
I20260807 08:36:27.227967 26822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d04dd94897dc42a5ae93c67e1cf47d4a" candidate_uuid: "5eb9d3a689dd469e843aae83c9959deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18365e75788745b7b74fb5fca9fbbd60"
I20260807 08:36:27.228143 26822 raft_consensus.cc:3037] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:36:27.229916 26822 raft_consensus.cc:2454] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5eb9d3a689dd469e843aae83c9959deb in term 1.
I20260807 08:36:27.230367 26936 leader_election.cc:302] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18365e75788745b7b74fb5fca9fbbd60, 5eb9d3a689dd469e843aae83c9959deb; no voters:
I20260807 08:36:27.230592 27136 raft_consensus.cc:2781] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:36:27.230713 27116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:27.215159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:42503 (local address 127.9.174.1:44921)
0807 08:36:27.215300 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 08:36:27.215304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:36:27.219897 (+ 4593us) server_negotiation.cc:407] Connection header received
0807 08:36:27.219977 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:36:27.219980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:36:27.220046 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:36:27.220117 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:36:27.222563 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:27.223409 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:27.229036 (+ 5627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:27.229716 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:27.229842 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:27.229942 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 08:36:27.230397 (+ 455us) server_negotiation.cc:299] Negotiation successful
0807 08:36:27.230548 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 08:36:27.230865 27136 raft_consensus.cc:686] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [term 1 LEADER]: Becoming Leader. State: Replica: 5eb9d3a689dd469e843aae83c9959deb, State: Running, Role: LEADER
I20260807 08:36:27.231382 27136 consensus_queue.cc:237] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb9d3a689dd469e843aae83c9959deb" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45967 } } peers { permanent_uuid: "18365e75788745b7b74fb5fca9fbbd60" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43363 } } peers { permanent_uuid: "81744ac4b825447a9ecf3dc9696148fe" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 44921 } }
I20260807 08:36:27.234046 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:27.213761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:44921 (local address 127.9.174.3:42503)
0807 08:36:27.219388 (+ 5627us) negotiation.cc:107] Waiting for socket to connect
0807 08:36:27.219408 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:36:27.219549 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:36:27.222226 (+ 2677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:36:27.222229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:36:27.222255 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:36:27.222404 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:36:27.222410 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:36:27.223707 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:36:27.223711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:36:27.228864 (+ 5153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:36:27.228877 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:36:27.230111 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:36:27.230116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:36:27.230225 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:27.230253 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 08:36:27.231183 (+ 930us) client_negotiation.cc:240] Negotiation successful
0807 08:36:27.231281 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5528,"thread_start_us":203,"threads_started":1}
I20260807 08:36:27.237942 26615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d04dd94897dc42a5ae93c67e1cf47d4a" candidate_uuid: "5eb9d3a689dd469e843aae83c9959deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81744ac4b825447a9ecf3dc9696148fe" is_pre_election: true
I20260807 08:36:27.238399 26615 raft_consensus.cc:2454] T d04dd94897dc42a5ae93c67e1cf47d4a P 81744ac4b825447a9ecf3dc9696148fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5eb9d3a689dd469e843aae83c9959deb in term 0.
I20260807 08:36:27.238921 26613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d04dd94897dc42a5ae93c67e1cf47d4a" candidate_uuid: "5eb9d3a689dd469e843aae83c9959deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81744ac4b825447a9ecf3dc9696148fe"
I20260807 08:36:27.239060 26613 raft_consensus.cc:3037] T d04dd94897dc42a5ae93c67e1cf47d4a P 81744ac4b825447a9ecf3dc9696148fe [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:36:27.241199 26613 raft_consensus.cc:2454] T d04dd94897dc42a5ae93c67e1cf47d4a P 81744ac4b825447a9ecf3dc9696148fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5eb9d3a689dd469e843aae83c9959deb in term 1.
I20260807 08:36:27.243855 26334 catalog_manager.cc:5495] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5eb9d3a689dd469e843aae83c9959deb (127.9.174.3). New cstate: current_term: 1 leader_uuid: "5eb9d3a689dd469e843aae83c9959deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb9d3a689dd469e843aae83c9959deb" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 45967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18365e75788745b7b74fb5fca9fbbd60" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81744ac4b825447a9ecf3dc9696148fe" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 44921 } health_report { overall_health: UNKNOWN } } }
I20260807 08:36:27.291190 27113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:27.284910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41246 (local address 127.9.174.3:45967)
0807 08:36:27.285454 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0807 08:36:27.285460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:36:27.285477 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:36:27.285606 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:36:27.285609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:36:27.285698 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:36:27.285820 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:36:27.286268 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:27.287167 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:27.288505 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:36:27.288856 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:36:27.288970 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:36:27.289448 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:36:27.290255 (+ 807us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:36:27.290387 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 08:36:27.290889 (+ 502us) server_negotiation.cc:299] Negotiation successful
0807 08:36:27.290974 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438}
I20260807 08:36:27.313169 26613 raft_consensus.cc:1261] T d04dd94897dc42a5ae93c67e1cf47d4a P 81744ac4b825447a9ecf3dc9696148fe [term 1 FOLLOWER]: Refusing update from remote peer 5eb9d3a689dd469e843aae83c9959deb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:36:27.314294 27136 consensus_queue.cc:1035] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "81744ac4b825447a9ecf3dc9696148fe" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 44921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:36:27.319861 26822 raft_consensus.cc:1261] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60 [term 1 FOLLOWER]: Refusing update from remote peer 5eb9d3a689dd469e843aae83c9959deb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:36:27.320771 27136 consensus_queue.cc:1035] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "18365e75788745b7b74fb5fca9fbbd60" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:36:27.343673 27146 mvcc.cc:205] Tried to move back new op lower bound from 7315831960804446208 to 7315831960504287232. Current Snapshot: MvccSnapshot[applied={T|T < 7315831960804446208}]
I20260807 08:36:27.359117 27149 mvcc.cc:205] Tried to move back new op lower bound from 7315831960804446208 to 7315831960504287232. Current Snapshot: MvccSnapshot[applied={T|T < 7315831960804446208}]
I20260807 08:36:27.360806 27147 mvcc.cc:205] Tried to move back new op lower bound from 7315831960804446208 to 7315831960504287232. Current Snapshot: MvccSnapshot[applied={T|T < 7315831960804446208}]
I20260807 08:36:27.361433 26334 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:36:27.361675 26334 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:36:27.365152 26334 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f931d94846f4dd392de1ad304728744: Sending DeleteTablet for 3 replicas of tablet d04dd94897dc42a5ae93c67e1cf47d4a
I20260807 08:36:27.366463 26997 tablet_service.cc:1514] Processing DeleteTablet for tablet d04dd94897dc42a5ae93c67e1cf47d4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:36:27 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:41234
I20260807 08:36:27.367681 26792 tablet_service.cc:1514] Processing DeleteTablet for tablet d04dd94897dc42a5ae93c67e1cf47d4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:36:27 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:49002
I20260807 08:36:27.368153 26582 tablet_service.cc:1514] Processing DeleteTablet for tablet d04dd94897dc42a5ae93c67e1cf47d4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:36:27 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:39114
I20260807 08:36:27.369515 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 26466
I20260807 08:36:27.383339 27159 tablet_replica.cc:330] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60: stopping tablet replica
I20260807 08:36:27.383929 27159 raft_consensus.cc:2229] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:36:27.384465 27159 raft_consensus.cc:2258] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:36:27.389640 27159 ts_tablet_manager.cc:1905] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:36:27.393394 27159 ts_tablet_manager.cc:1918] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:36:27.393528 27159 log.cc:1196] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/d04dd94897dc42a5ae93c67e1cf47d4a
I20260807 08:36:27.393872 27159 ts_tablet_manager.cc:1939] T d04dd94897dc42a5ae93c67e1cf47d4a P 18365e75788745b7b74fb5fca9fbbd60: Deleting consensus metadata
I20260807 08:36:27.411981 26318 catalog_manager.cc:4841] TS 18365e75788745b7b74fb5fca9fbbd60 (127.9.174.2:43363): tablet d04dd94897dc42a5ae93c67e1cf47d4a (table test-table [id=25940e6ceb7746378bddb4fd97ebc1c8]) successfully deleted
I20260807 08:36:27.413219 27157 tablet_replica.cc:330] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb: stopping tablet replica
I20260807 08:36:27.413779 27157 raft_consensus.cc:2229] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:36:27.414364 27157 raft_consensus.cc:2258] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:36:27.416590 27157 ts_tablet_manager.cc:1905] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:36:27.422953 27157 ts_tablet_manager.cc:1918] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:36:27.423162 27157 log.cc:1196] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/d04dd94897dc42a5ae93c67e1cf47d4a
I20260807 08:36:27.423569 27157 ts_tablet_manager.cc:1939] T d04dd94897dc42a5ae93c67e1cf47d4a P 5eb9d3a689dd469e843aae83c9959deb: Deleting consensus metadata
I20260807 08:36:27.425302 26317 catalog_manager.cc:4841] TS 5eb9d3a689dd469e843aae83c9959deb (127.9.174.3:45967): tablet d04dd94897dc42a5ae93c67e1cf47d4a (table test-table [id=25940e6ceb7746378bddb4fd97ebc1c8]) successfully deleted
W20260807 08:36:27.425925 26318 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:44921 (error 108)
I20260807 08:36:27.427223 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 26696
I20260807 08:36:27.459472 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:36:27.458919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:44921 (local address 127.0.0.1:39122)
0807 08:36:27.459220 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 08:36:27.459316 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190}
W20260807 08:36:27.459772 26318 catalog_manager.cc:4584] TS 81744ac4b825447a9ecf3dc9696148fe (127.9.174.1:44921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d04dd94897dc42a5ae93c67e1cf47d4a: Network error: Client connection negotiation failed: client connection to 127.9.174.1:44921: connect: Connection refused (error 111)
I20260807 08:36:27.460987 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 26907
I20260807 08:36:27.490078 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 26293
I20260807 08:36:27.509125 9912 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 08:36:27.868795 9912 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 08:36:27.868974 9912 mini_ranger.cc:66] Stopping Ranger...
I20260807 08:36:28.225471 9912 mini_ranger.cc:68] Stopped Ranger
2026-08-07 08:36:28.225 UTC [25646] LOG: received smart shutdown request
2026-08-07 08:36:28.231 UTC [25646] LOG: background worker "logical replication launcher" (PID 25659) exited with exit code 1
2026-08-07 08:36:28.231 UTC [25651] LOG: shutting down
2026-08-07 08:36:28.429 UTC [25646] LOG: database system is shut down
2026-08-07T08:36:28Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (57065 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:36:28 dist-test-slave-h3ft krb5kdc[27178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:36:28 dist-test-slave-h3ft krb5kdc[27178](info): set up 2 sockets
Aug 07 08:36:28 dist-test-slave-h3ft krb5kdc[27178](info): commencing operation
krb5kdc: starting...
W20260807 08:36:29.923179 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.000s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:36:30 dist-test-slave-h3ft krb5kdc[27178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:36:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:36:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.174.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.174.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.174.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/rangeradmin_127.9.174.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.174.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/rangeradmin_127.9.174.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.174.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.174.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.174.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/rangerlookup_127.9.174.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.174.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/rangerlookup_127.9.174.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.174.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.174.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.174.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/HTTP_127.9.174.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.174.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/HTTP_127.9.174.20@KRBTEST.COM.keytab.
I20260807 08:36:30.164053 9912 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task1t_l5C/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/postgres -l logfile start
2026-08-07 08:36:33.108 UTC [27222] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 08:36:33.108 UTC [27222] LOG: listening on IPv4 address "127.9.174.20", port 46105
2026-08-07 08:36:33.113 UTC [27222] LOG: listening on Unix socket "/tmp/.s.PGSQL.46105"
2026-08-07 08:36:33.140 UTC [27227] LOG: database system was shut down at 2026-08-07 08:36:31 UTC
2026-08-07 08:36:33.144 UTC [27222] LOG: database system is ready to accept connections
I20260807 08:36:34.348767 9912 mini_postgres.cc:96] Postgres bound to 46105
2026-08-07 08:36:34.359 UTC [27239] FATAL: database "slave" does not exist
127.9.174.20:46105 - accepting connections
I20260807 08:36:34.360193 9912 mini_ranger.cc:161] Starting Ranger...
I20260807 08:36:34.379902 9912 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 08:36:34.943693 9912 mini_ranger.cc:87] Created ranger Postgres database
I20260807 08:36:34.943816 9912 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-admin
2026-08-07 08:36:35,262 [I] DB FLAVOR :POSTGRES
2026-08-07 08:36:35,263 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 08:36:35,263 [I] Checking connection..
2026-08-07 08:36:35,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 08:36:35,557 [I] Checking connection passed.
2026-08-07 08:36:35,558 [I] --------- Verifying version history table ---------
2026-08-07 08:36:35,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:36:35,848 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 08:36:35,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:36:36,172 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 08:36:36,172 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 08:36:36,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 08:36:36.443 UTC [27344] WARNING: there is no transaction in progress
2026-08-07 08:36:36,455 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 08:36:36,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:36:36,753 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 08:36:36,753 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 08:36:36,753 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 08:36:37,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 08:36:37,320 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h3ft.c.gcp-upstream.internal','N') ;"
2026-08-07 08:36:37,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 08:36:37,907 [I] Table x_portal_user does not exist in database ranger
2026-08-07 08:36:37,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 08:36:38,209 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 08:36:38,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 08:36:38,478 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 08:36:38,790 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 08:36:38,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 08:36:39.919 UTC [27512] WARNING: there is no transaction in progress
2026-08-07 08:36:39.940 UTC [27512] WARNING: there is no transaction in progress
2026-08-07 08:36:39.962 UTC [27512] WARNING: there is no transaction in progress
2026-08-07 08:36:39.982 UTC [27512] WARNING: there is no transaction in progress
2026-08-07 08:36:40.004 UTC [27512] WARNING: there is no transaction in progress
2026-08-07 08:36:40.028 UTC [27512] WARNING: there is no transaction in progress
2026-08-07 08:36:40.105 UTC [27512] WARNING: there is no transaction in progress
2026-08-07 08:36:40.115 UTC [27512] WARNING: there is no transaction in progress
2026-08-07 08:36:40.134 UTC [27512] WARNING: there is no transaction in progress
2026-08-07 08:36:40.145 UTC [27512] WARNING: there is no transaction in progress
2026-08-07 08:36:40.158 UTC [27512] WARNING: there is no transaction in progress
2026-08-07 08:36:40.175 UTC [27512] WARNING: there is no transaction in progress
2026-08-07 08:36:40.188 UTC [27512] WARNING: there is no transaction in progress
2026-08-07 08:36:40.195 UTC [27512] WARNING: there is no transaction in progress
2026-08-07 08:36:40.204 UTC [27512] WARNING: there is no transaction in progress
2026-08-07 08:36:40.212 UTC [27512] WARNING: there is no transaction in progress
2026-08-07 08:36:40,961 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 08:36:40,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 08:36:41,238 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 08:36:41,238 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 08:36:41,544 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 08:36:41,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 08:36:41,834 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 08:36:41,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 08:36:42,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 08:36:42,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h3ft.c.gcp-upstream.internal';"
2026-08-07 08:36:42,734 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 08:36:42,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 08:36:43,035 [I] DB_PATCHES have already been applied
I20260807 08:36:43.042572 9912 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-admin:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task1t_l5C/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 08:36:43.042662 9912 mini_ranger.cc:193] Using host: 127.9.174.20
I20260807 08:36:43.048473 9912 mini_ranger.cc:227] Ranger admin URL: http://127.9.174.20:40987
Aug 07, 2026 8:36:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:36:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:36:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 8:36:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 8:36:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 8:36:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 8:36:44 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 08:36:44.445027 9912 mini_ranger.cc:160] Time spent starting Ranger: real 10.085s user 0.001s sys 0.014s
Aug 07, 2026 8:37:08 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 8:37:08 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 8:37:08 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 8:37:08 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 8:37:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 8:37:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 8:37:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 8:37:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 8:37:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 8:37:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 8:37:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 8:37:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 8:37:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 8:37:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 8:37:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 8:37:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 8:37:08 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 8:37:09 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 08:37:10.080541 9912 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.174.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.174.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.174.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/rangerkms_127.9.174.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.174.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/rangerkms_127.9.174.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.174.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.174.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.174.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/HTTP_127.9.174.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.174.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/HTTP_127.9.174.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 08:37:10 dist-test-slave-h3ft krb5kdc[27178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091830, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 08:37:10.206902 9912 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 08:37:10.223991 9912 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 08:37:11.285753 9912 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 08:37:11.285897 9912 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-kms
I20260807 08:37:11.285924 9912 mini_ranger_kms.cc:227] Using postgres at 127.9.174.20:46105
2026-08-07 08:37:11,519 [I] DB FLAVOR :POSTGRES
2026-08-07 08:37:11,520 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 08:37:11,520 [I] Checking connection
2026-08-07 08:37:11,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 08:37:11,807 [I] connection success
2026-08-07 08:37:11,807 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 08:37:11,807 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 08:37:11,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 08:37:12,083 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 08:37:12,083 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 08:37:12,083 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 08:37:12,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.174.20:46105/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 08:37:12,417 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 08:37:13.031236 9912 mini_ranger_kms.cc:310] Created kms service
I20260807 08:37:13.251677 9912 mini_ranger_kms.cc:326] Created kudu user
I20260807 08:37:13.303252 9912 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 08:37:13 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091830, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
I20260807 08:37:13.813556 9912 mini_ranger_kms.cc:384] Added ranger policy
I20260807 08:37:13.813779 9912 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-kms:/tmp/dist-test-task1t_l5C/build/debug/bin/postgresql.jar:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task1t_l5C/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 08:37:13.813804 9912 mini_ranger_kms.cc:240] Using host: 127.9.174.20
I20260807 08:37:13.821224 9912 mini_ranger_kms.cc:276] Ranger KMS PID: 27781
I20260807 08:37:13.821384 9912 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.174.20:48797
Aug 07, 2026 8:37:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:37:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:37:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 8:37:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 8:37:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 8:37:15 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 08:37:16 dist-test-slave-h3ft krb5kdc[27178](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786091836, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.174.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:16.556886 9912 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.350s user 0.016s sys 0.201s
I20260807 08:37:16.557070 9912 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 08:37:16.557161 9912 mini_ranger_kms.cc:401] 127.9.174.20:48797/kms/v1/keys
Aug 07 08:37:16 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091836, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.174.20@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
Aug 07 08:37:17 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091836, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.174.20@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
Aug 07 08:37:20 dist-test-slave-h3ft krb5kdc[27178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:37:20.170950 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:44217
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:37413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:44217
--ranger_config_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.174.20:48797/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:20.304276 27852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:20.304590 27852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:20.304632 27852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:20.308670 27852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:20.308733 27852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:20.308753 27852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:20.308769 27852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:20.308784 27852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:20.314489 27852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:37413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.174.20:48797/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:44217
--ranger_config_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:44217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:20.315829 27852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:20.317096 27852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:37:20.327453 27852 server_base.cc:1034] running on GCE node
W20260807 08:37:20.327374 27868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:20.327497 27863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:20.327391 27864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:20.328302 27852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:20.329711 27852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:20.330906 27852 hybrid_clock.cc:648] HybridClock initialized: now 1786091840330894 us; error 37 us; skew 500 ppm
Aug 07 08:37:20 dist-test-slave-h3ft krb5kdc[27178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091840, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:20.334100 27852 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:37:20.335144 27852 webserver.cc:460] Webserver started at http://127.9.174.62:42909/ using document root <none> and password file <none>
I20260807 08:37:20.335758 27852 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:20.335815 27852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:20.336010 27852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:37:20 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091840, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
I20260807 08:37:20.399045 27852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "219b893e21ce43c48f4d03c10305fa12"
format_stamp: "Formatted at 2026-08-07 08:37:20 on dist-test-slave-h3ft"
server_key: "c5bdedd31a6a48b91237bbcd0630d0b2"
server_key_iv: "fa405f2dd69e048024cc227b8e1a8a2a"
server_key_version: "kuduclusterkey@0"
I20260807 08:37:20.399986 27852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "219b893e21ce43c48f4d03c10305fa12"
format_stamp: "Formatted at 2026-08-07 08:37:20 on dist-test-slave-h3ft"
server_key: "c5bdedd31a6a48b91237bbcd0630d0b2"
server_key_iv: "fa405f2dd69e048024cc227b8e1a8a2a"
server_key_version: "kuduclusterkey@0"
I20260807 08:37:20.405990 27852 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 08:37:20.425855 27879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:20.428047 27852 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:37:20.428233 27852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "219b893e21ce43c48f4d03c10305fa12"
format_stamp: "Formatted at 2026-08-07 08:37:20 on dist-test-slave-h3ft"
server_key: "c5bdedd31a6a48b91237bbcd0630d0b2"
server_key_iv: "fa405f2dd69e048024cc227b8e1a8a2a"
server_key_version: "kuduclusterkey@0"
I20260807 08:37:20.428445 27852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:20.450264 27852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:20.454623 27852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:20.454878 27852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:20.465608 27852 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:44217
I20260807 08:37:20.465628 27952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:44217 every 8 connection(s)
I20260807 08:37:20.467190 27852 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:20.470958 27954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:20.472424 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 27852
I20260807 08:37:20.472609 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:37:20.473038 9912 external_mini_cluster.cc:1428] Setting key ef97c7f930406293381d91e72c1afa98
I20260807 08:37:20.479597 27954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12: Bootstrap starting.
I20260807 08:37:20.482961 27954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:20.484318 27954 log.cc:824] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:37:20 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:20.487310 27954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12: No bootstrap required, opened a new log
I20260807 08:37:20.490780 27954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219b893e21ce43c48f4d03c10305fa12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 44217 } }
I20260807 08:37:20.491132 27954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:20.491223 27954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 219b893e21ce43c48f4d03c10305fa12, State: Initialized, Role: FOLLOWER
I20260807 08:37:20.491848 27954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219b893e21ce43c48f4d03c10305fa12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 44217 } }
I20260807 08:37:20.492039 27954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:20.492149 27954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:20.492280 27954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:20.492795 27961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.475059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44948 (local address 127.9.174.62:44217)
0807 08:37:20.475682 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:20.475707 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:20.475764 (+ 57us) server_negotiation.cc:407] Connection header received
0807 08:37:20.476823 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:20.476854 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:20.477416 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:20.477873 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:20.478975 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.480702 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.482059 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.482864 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.483126 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.486619 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:20.486626 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:20.486645 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:20.486676 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:20.489360 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:20.489856 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:20.489862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:20.489867 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:20.489985 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:20.490294 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:20.490298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:20.490300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:20.490773 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:20.491041 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:20.491350 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 08:37:20.491704 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":180,"threads_started":1}
I20260807 08:37:20.493593 27954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219b893e21ce43c48f4d03c10305fa12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 44217 } }
I20260807 08:37:20.494046 27954 leader_election.cc:302] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 219b893e21ce43c48f4d03c10305fa12; no voters:
I20260807 08:37:20.494505 27954 leader_election.cc:288] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:20.494604 27963 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:20.494876 27963 raft_consensus.cc:686] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 1 LEADER]: Becoming Leader. State: Replica: 219b893e21ce43c48f4d03c10305fa12, State: Running, Role: LEADER
I20260807 08:37:20.495329 27963 consensus_queue.cc:237] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219b893e21ce43c48f4d03c10305fa12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 44217 } }
I20260807 08:37:20.496289 27954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:20.496843 27963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 219b893e21ce43c48f4d03c10305fa12. Latest consensus state: current_term: 1 leader_uuid: "219b893e21ce43c48f4d03c10305fa12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219b893e21ce43c48f4d03c10305fa12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 44217 } } }
I20260807 08:37:20.497141 27963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:20.497437 27964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "219b893e21ce43c48f4d03c10305fa12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219b893e21ce43c48f4d03c10305fa12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 44217 } } }
I20260807 08:37:20.497570 27964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:20.497756 27970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:20.501636 27970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:20.502645 27954 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 08:37:20.513505 27970 catalog_manager.cc:1347] Generated new cluster ID: c118594d3631493186b57b6588eb9388
I20260807 08:37:20.513607 27970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:20.542745 27970 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:20.544461 27970 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:20.555514 27970 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12: Generated new TSK 0
I20260807 08:37:20.556710 27970 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:37:21.334479 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.174.20:48797/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:44217
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:37413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:21.481626 28019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:21.481969 28019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:21.482065 28019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:21.486618 28019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:21.486711 28019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:21.486804 28019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:37:21.494920 28019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:37413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.174.20:48797/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:44217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:21.496821 28019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:21.498493 28019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:21.512130 28032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:21.512317 28028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:21.512428 28030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:21.513216 28019 server_base.cc:1034] running on GCE node
I20260807 08:37:21.513878 28019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:21.514894 28019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:21.516108 28019 hybrid_clock.cc:648] HybridClock initialized: now 1786091841516082 us; error 39 us; skew 500 ppm
Aug 07 08:37:21 dist-test-slave-h3ft krb5kdc[27178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091841, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:21.520363 28019 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:37:21.521552 28019 webserver.cc:460] Webserver started at http://127.9.174.1:40799/ using document root <none> and password file <none>
I20260807 08:37:21.522292 28019 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:21.522400 28019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:21.522712 28019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:37:21 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091841, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
I20260807 08:37:21.543941 28019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "0bc1fa66524d4b13bb2f40b7880e0048"
format_stamp: "Formatted at 2026-08-07 08:37:21 on dist-test-slave-h3ft"
server_key: "1e042df00d3ccd8221306120b388f08e"
server_key_iv: "281fa25b04477e021297b31022f5852e"
server_key_version: "kuduclusterkey@0"
I20260807 08:37:21.544900 28019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "0bc1fa66524d4b13bb2f40b7880e0048"
format_stamp: "Formatted at 2026-08-07 08:37:21 on dist-test-slave-h3ft"
server_key: "1e042df00d3ccd8221306120b388f08e"
server_key_iv: "281fa25b04477e021297b31022f5852e"
server_key_version: "kuduclusterkey@0"
I20260807 08:37:21.554188 28019 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.002s sys 0.005s
I20260807 08:37:21.581439 28041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:21.583495 28019 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:37:21.583664 28019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "0bc1fa66524d4b13bb2f40b7880e0048"
format_stamp: "Formatted at 2026-08-07 08:37:21 on dist-test-slave-h3ft"
server_key: "1e042df00d3ccd8221306120b388f08e"
server_key_iv: "281fa25b04477e021297b31022f5852e"
server_key_version: "kuduclusterkey@0"
I20260807 08:37:21.583833 28019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:21.599367 28019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:21.603633 28019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:21.603986 28019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:21.604957 28019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:21.606319 28019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:21.606433 28019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 08:37:21.606590 28019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:21.606738 28019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:21.620401 28019 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:40937
I20260807 08:37:21.622067 28019 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:21.623247 28218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:40937 every 8 connection(s)
I20260807 08:37:21.627684 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 28019
I20260807 08:37:21.627833 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:37:21.628202 9912 external_mini_cluster.cc:1428] Setting key 342e07da2716e7a80b1a4b0a99a2daa4
Aug 07 08:37:21 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091841, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:21.646325 28223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:21.627316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:46801 (local address 127.9.174.62:44217)
0807 08:37:21.630879 (+ 3563us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:21.630886 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:21.630904 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:37:21.630970 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:21.630975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:21.631084 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:21.631201 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:21.633619 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:21.634565 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:21.636476 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:21.636837 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:21.636954 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:21.640730 (+ 3776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:21.640735 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:21.640738 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:21.640772 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:21.642974 (+ 2202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:21.643762 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:21.643766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:21.643769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:21.643882 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:21.644402 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:21.644405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:21.644407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:21.644827 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:21.644946 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:21.645834 (+ 888us) server_negotiation.cc:299] Negotiation successful
0807 08:37:21.646036 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3454,"thread_start_us":123,"threads_started":1}
I20260807 08:37:21.648934 28224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:21.626670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:44217 (local address 127.9.174.1:46801)
0807 08:37:21.629106 (+ 2436us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:21.629172 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:21.630355 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:21.631607 (+ 1252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:21.631622 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:21.632111 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:21.633398 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:21.633430 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:21.634730 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:21.634734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:21.636323 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:21.636336 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:21.637195 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:21.637199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:21.637442 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:21.638320 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:21.638345 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:21.640479 (+ 2134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:21.643175 (+ 2696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:21.643182 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:21.643206 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:21.643631 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:21.644010 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:21.644014 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:21.644016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:21.644243 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:21.645013 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:21.645023 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:21.645494 (+ 471us) client_negotiation.cc:769] Sending connection context
0807 08:37:21.647035 (+ 1541us) client_negotiation.cc:240] Negotiation successful
0807 08:37:21.647500 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2151,"thread_start_us":724,"threads_started":1}
I20260807 08:37:21.650966 28219 heartbeater.cc:347] Connected to a master server at 127.9.174.62:44217
I20260807 08:37:21.651322 28219 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:21.652146 28219 heartbeater.cc:511] Master 127.9.174.62:44217 requested a full tablet report, sending...
I20260807 08:37:21.654588 27906 ts_manager.cc:194] Registered new tserver with Master: 0bc1fa66524d4b13bb2f40b7880e0048 (127.9.174.1:40937)
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
I20260807 08:37:21.656400 27906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:46801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:37:21.679137 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.174.20:48797/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:44217
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:37413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:37:21 dist-test-slave-h3ft krb5kdc[27178](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786091841, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 08:37:21.868239 28228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:21.868808 28228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:21.868981 28228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:21.876353 28228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:21.876480 28228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:21.876592 28228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:37:21.886338 28228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:37413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.174.20:48797/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:44217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:21.888597 28228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:21.890458 28228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:21.906749 28239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:21.906749 28248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:21.907454 28238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:21.908720 28228 server_base.cc:1034] running on GCE node
I20260807 08:37:21.909409 28228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:21.910475 28228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:21.911921 28228 hybrid_clock.cc:648] HybridClock initialized: now 1786091841911882 us; error 50 us; skew 500 ppm
Aug 07 08:37:21 dist-test-slave-h3ft krb5kdc[27178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091841, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:21.916661 28228 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:37:21.918111 28228 webserver.cc:460] Webserver started at http://127.9.174.2:32885/ using document root <none> and password file <none>
I20260807 08:37:21.919045 28228 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:21.919231 28228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:21.919590 28228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:37:21 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091841, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
I20260807 08:37:21.941200 28228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "b7db396ccda041ec8f51b3914ddcedb9"
format_stamp: "Formatted at 2026-08-07 08:37:21 on dist-test-slave-h3ft"
server_key: "790c1855ea7ec91895c5c2c8187d487d"
server_key_iv: "b35cea9224519f374770ba22ef59193a"
server_key_version: "kuduclusterkey@0"
I20260807 08:37:21.943156 28228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "b7db396ccda041ec8f51b3914ddcedb9"
format_stamp: "Formatted at 2026-08-07 08:37:21 on dist-test-slave-h3ft"
server_key: "790c1855ea7ec91895c5c2c8187d487d"
server_key_iv: "b35cea9224519f374770ba22ef59193a"
server_key_version: "kuduclusterkey@0"
I20260807 08:37:21.957639 28228 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.006s sys 0.000s
I20260807 08:37:21.989367 28256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:21.992715 28228 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.004s sys 0.001s
I20260807 08:37:21.993018 28228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "b7db396ccda041ec8f51b3914ddcedb9"
format_stamp: "Formatted at 2026-08-07 08:37:21 on dist-test-slave-h3ft"
server_key: "790c1855ea7ec91895c5c2c8187d487d"
server_key_iv: "b35cea9224519f374770ba22ef59193a"
server_key_version: "kuduclusterkey@0"
I20260807 08:37:21.993229 28228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:22.015738 28228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:22.019685 28228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:22.019942 28228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:22.020721 28228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:22.021821 28228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:22.021921 28228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:22.022042 28228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:22.022104 28228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:22.037356 28228 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:36749
I20260807 08:37:22.039163 28228 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:22.043963 28436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:36749 every 8 connection(s)
I20260807 08:37:22.045830 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 28228
I20260807 08:37:22.045990 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:37:22.046430 9912 external_mini_cluster.cc:1428] Setting key 5326327fc054e332bfefe8e232576257
Aug 07 08:37:22 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091841, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:22.065984 28223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:22.047458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:36385 (local address 127.9.174.62:44217)
0807 08:37:22.047876 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:22.047881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:22.049479 (+ 1598us) server_negotiation.cc:407] Connection header received
0807 08:37:22.050979 (+ 1500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:22.050984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:22.051071 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:22.051132 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:22.053140 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:22.054110 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:22.056121 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:22.056525 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:22.056615 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:22.060662 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:22.060668 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:22.060671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:22.060715 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:22.062800 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:22.063648 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:22.063653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:22.063655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:22.063762 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:22.064223 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:22.064226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:22.064228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:22.064531 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:22.064625 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:22.065557 (+ 932us) server_negotiation.cc:299] Negotiation successful
0807 08:37:22.065744 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291}
I20260807 08:37:22.068674 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:22.047770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:44217 (local address 127.9.174.2:36385)
0807 08:37:22.049287 (+ 1517us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:22.049354 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:22.050677 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:22.051649 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:22.051659 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:22.052102 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:22.052942 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:22.052965 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:22.054291 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:22.054295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:22.055958 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:22.055971 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:22.056894 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:22.056899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:22.057140 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:22.058051 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:22.058077 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:22.060325 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:22.062974 (+ 2649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:22.062985 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:22.063039 (+ 54us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:22.063509 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:22.063877 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:22.063880 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:22.063882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:22.064104 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:22.064703 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:22.064712 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:22.065188 (+ 476us) client_negotiation.cc:769] Sending connection context
0807 08:37:22.066666 (+ 1478us) client_negotiation.cc:240] Negotiation successful
0807 08:37:22.067191 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1240,"thread_start_us":211,"threads_started":1}
I20260807 08:37:22.072216 28438 heartbeater.cc:347] Connected to a master server at 127.9.174.62:44217
I20260807 08:37:22.072570 28438 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:22.073437 28438 heartbeater.cc:511] Master 127.9.174.62:44217 requested a full tablet report, sending...
I20260807 08:37:22.075166 27906 ts_manager.cc:194] Registered new tserver with Master: b7db396ccda041ec8f51b3914ddcedb9 (127.9.174.2:36749)
I20260807 08:37:22.076136 27906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:36385
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:37:22.124703 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.174.20:48797/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:44217
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:37413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:37:22 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091841, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
W20260807 08:37:22.302114 28446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:22.302676 28446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:22.302767 28446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:22.309031 28446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:22.309156 28446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:22.309268 28446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:37:22.317974 28446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:37413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.174.20:48797/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:44217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:22.320165 28446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:22.322029 28446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:22.338675 28453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:22.339622 28457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:22.340263 28446 server_base.cc:1034] running on GCE node
W20260807 08:37:22.340044 28452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:22.341207 28446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:22.342224 28446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:22.345541 28446 hybrid_clock.cc:648] HybridClock initialized: now 1786091842345443 us; error 99 us; skew 500 ppm
Aug 07 08:37:22 dist-test-slave-h3ft krb5kdc[27178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091842, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:22.350281 28446 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:37:22.351902 28446 webserver.cc:460] Webserver started at http://127.9.174.3:39969/ using document root <none> and password file <none>
I20260807 08:37:22.352941 28446 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:22.353111 28446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:22.353507 28446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:37:22 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091842, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
I20260807 08:37:22.389961 28446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "304a34e4aa76450a9e542e5a3c184780"
format_stamp: "Formatted at 2026-08-07 08:37:22 on dist-test-slave-h3ft"
server_key: "5f06b3da7c37c60e997fa1ac29d259cf"
server_key_iv: "1a0659eae8e0db1847623ab9d44fdf3d"
server_key_version: "kuduclusterkey@0"
I20260807 08:37:22.391214 28446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "304a34e4aa76450a9e542e5a3c184780"
format_stamp: "Formatted at 2026-08-07 08:37:22 on dist-test-slave-h3ft"
server_key: "5f06b3da7c37c60e997fa1ac29d259cf"
server_key_iv: "1a0659eae8e0db1847623ab9d44fdf3d"
server_key_version: "kuduclusterkey@0"
I20260807 08:37:22.398280 28446 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260807 08:37:22.419400 28468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:22.421226 28446 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:37:22.421413 28446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "304a34e4aa76450a9e542e5a3c184780"
format_stamp: "Formatted at 2026-08-07 08:37:22 on dist-test-slave-h3ft"
server_key: "5f06b3da7c37c60e997fa1ac29d259cf"
server_key_iv: "1a0659eae8e0db1847623ab9d44fdf3d"
server_key_version: "kuduclusterkey@0"
I20260807 08:37:22.421612 28446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:22.444871 28446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:22.449481 28446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:22.449862 28446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:22.450853 28446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:22.452224 28446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:22.452342 28446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 08:37:22.452451 28446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:22.452528 28446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:22.466487 28446 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:38337
I20260807 08:37:22.468076 28446 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:37:22.474323 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 28446
I20260807 08:37:22.474522 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:37:22.474988 9912 external_mini_cluster.cc:1428] Setting key 752c99f0561dec24b3558b8603f873e5
I20260807 08:37:22.479310 28637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:38337 every 8 connection(s)
Aug 07 08:37:22 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091842, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:22.498203 28223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:22.478889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:33179 (local address 127.9.174.62:44217)
0807 08:37:22.479068 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:22.479073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:22.480688 (+ 1615us) server_negotiation.cc:407] Connection header received
0807 08:37:22.482217 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:22.482221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:22.482293 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:22.482351 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:22.484422 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:22.485281 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:22.487318 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:22.487678 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:22.487769 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:22.492842 (+ 5073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:22.492847 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:22.492851 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:22.492888 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:22.495036 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:22.495820 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:22.495825 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:22.495827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:22.495938 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:22.496467 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:22.496471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:22.496473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:22.496783 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:22.496884 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:22.497801 (+ 917us) server_negotiation.cc:299] Negotiation successful
0807 08:37:22.497990 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 08:37:22.500530 28640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:22.477544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:44217 (local address 127.9.174.3:33179)
0807 08:37:22.480467 (+ 2923us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:22.480542 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:22.481892 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:22.482894 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:22.482905 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:22.483395 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:22.484220 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:22.484241 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:22.485513 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:22.485517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:22.487158 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:22.487171 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:22.488055 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:22.488059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:22.488287 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:22.490348 (+ 2061us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:22.490376 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:22.492552 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:22.495186 (+ 2634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:22.495194 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:22.495221 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:22.495671 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:22.496065 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:22.496068 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:22.496071 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:22.496314 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:22.496970 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:22.496979 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:22.497473 (+ 494us) client_negotiation.cc:769] Sending connection context
0807 08:37:22.498538 (+ 1065us) client_negotiation.cc:240] Negotiation successful
0807 08:37:22.499067 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2620,"thread_start_us":187,"threads_started":1}
I20260807 08:37:22.503794 28638 heartbeater.cc:347] Connected to a master server at 127.9.174.62:44217
I20260807 08:37:22.504153 28638 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:22.505017 28638 heartbeater.cc:511] Master 127.9.174.62:44217 requested a full tablet report, sending...
I20260807 08:37:22.506491 27906 ts_manager.cc:194] Registered new tserver with Master: 304a34e4aa76450a9e542e5a3c184780 (127.9.174.3:38337)
I20260807 08:37:22.507444 27906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:33179
I20260807 08:37:22.515065 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:37:22.516258 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 28019
I20260807 08:37:22.536679 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 28228
I20260807 08:37:22.557577 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 28446
I20260807 08:37:22.574978 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 27852
I20260807 08:37:22.591500 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:44217
--webserver_interface=127.9.174.62
--webserver_port=42909
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:37413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:44217
--ranger_config_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.174.20:48797/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.174.20:48797/kms with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:22.725874 28642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:22.726228 28642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:22.726271 28642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:22.730546 28642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:22.730628 28642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:22.730647 28642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:22.730691 28642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:22.730728 28642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:22.736717 28642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:37413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.174.20:48797/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:44217
--ranger_config_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:44217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=42909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:22.738224 28642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:22.739530 28642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:22.750013 28648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:22.750129 28642 server_base.cc:1034] running on GCE node
W20260807 08:37:22.750147 28652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:22.750021 28655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:22.751220 28642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:22.752574 28642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:22.753775 28642 hybrid_clock.cc:648] HybridClock initialized: now 1786091842753759 us; error 35 us; skew 500 ppm
Aug 07 08:37:22 dist-test-slave-h3ft krb5kdc[27178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091842, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:22.757254 28642 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:37:22.758502 28642 webserver.cc:460] Webserver started at http://127.9.174.62:42909/ using document root <none> and password file <none>
I20260807 08:37:22.759286 28642 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:22.759397 28642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 08:37:22 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091842, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
I20260807 08:37:22.814719 28642 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:37:22.817857 28668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:22.819643 28642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:22.819823 28642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "219b893e21ce43c48f4d03c10305fa12"
format_stamp: "Formatted at 2026-08-07 08:37:20 on dist-test-slave-h3ft"
server_key: "c5bdedd31a6a48b91237bbcd0630d0b2"
server_key_iv: "fa405f2dd69e048024cc227b8e1a8a2a"
server_key_version: "kuduclusterkey@0"
I20260807 08:37:22.820568 28642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:22.860918 28642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:22.865031 28642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:22.865312 28642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:22.874258 28642 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:44217
I20260807 08:37:22.874311 28741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:44217 every 8 connection(s)
I20260807 08:37:22.875717 28642 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:22.881309 28742 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:37:22.882540 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 28642
I20260807 08:37:22.883134 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.174.20:48797/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:40937
--local_ip_for_outbound_sockets=127.9.174.1
--tserver_master_addrs=127.9.174.62:44217
--webserver_port=40799
--webserver_interface=127.9.174.1
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:37413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.174.20:48797/kms with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:37:22.883664 28742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12: Bootstrap starting.
I20260807 08:37:22.904016 28742 log.cc:824] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:22.912612 28742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:37:22.913139 28742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12: Bootstrap complete.
I20260807 08:37:22.917951 28742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219b893e21ce43c48f4d03c10305fa12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 44217 } }
I20260807 08:37:22.918450 28742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 219b893e21ce43c48f4d03c10305fa12, State: Initialized, Role: FOLLOWER
I20260807 08:37:22.919144 28742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219b893e21ce43c48f4d03c10305fa12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 44217 } }
I20260807 08:37:22.919304 28742 raft_consensus.cc:388] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:22.919404 28742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:22.919514 28742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:37:22.921810 28742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219b893e21ce43c48f4d03c10305fa12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 44217 } }
I20260807 08:37:22.922200 28742 leader_election.cc:302] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 219b893e21ce43c48f4d03c10305fa12; no voters:
I20260807 08:37:22.922642 28742 leader_election.cc:288] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 08:37:22.922802 28748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:37:22.923106 28748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [term 2 LEADER]: Becoming Leader. State: Replica: 219b893e21ce43c48f4d03c10305fa12, State: Running, Role: LEADER
I20260807 08:37:22.923569 28748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219b893e21ce43c48f4d03c10305fa12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 44217 } }
I20260807 08:37:22.924226 28742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:22.925284 28749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "219b893e21ce43c48f4d03c10305fa12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219b893e21ce43c48f4d03c10305fa12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 44217 } } }
I20260807 08:37:22.925426 28749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:22.925778 28751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 219b893e21ce43c48f4d03c10305fa12. Latest consensus state: current_term: 2 leader_uuid: "219b893e21ce43c48f4d03c10305fa12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219b893e21ce43c48f4d03c10305fa12" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 44217 } } }
I20260807 08:37:22.925908 28751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:22.926824 28762 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:22.927316 28742 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 08:37:22.934401 28762 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:22.935398 28762 catalog_manager.cc:1259] Loaded cluster ID: c118594d3631493186b57b6588eb9388
I20260807 08:37:22.935506 28762 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:22.938838 28762 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:22.939756 28762 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12: Loaded TSK: 0
I20260807 08:37:22.941255 28762 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:37:23.051229 28746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:23.051887 28746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:23.052062 28746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:23.058156 28746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:23.058290 28746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:23.058415 28746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:37:23.065183 28746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:37413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.174.20:48797/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:40937
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=40799
--tserver_master_addrs=127.9.174.62:44217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:23.066751 28746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:23.068436 28746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:37:23.082436 28746 server_base.cc:1034] running on GCE node
W20260807 08:37:23.082690 28799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:23.082437 28800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:23.083524 28802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:23.084165 28746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:23.085357 28746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:23.088701 28746 hybrid_clock.cc:648] HybridClock initialized: now 1786091843088588 us; error 124 us; skew 500 ppm
Aug 07 08:37:23 dist-test-slave-h3ft krb5kdc[27178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091843, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:23.093441 28746 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:37:23.094971 28746 webserver.cc:460] Webserver started at http://127.9.174.1:40799/ using document root <none> and password file <none>
I20260807 08:37:23.095990 28746 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:23.096125 28746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 08:37:23 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091843, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
I20260807 08:37:23.140892 28746 fs_manager.cc:709] Time spent opening directory manager: real 0.018s user 0.004s sys 0.000s
I20260807 08:37:23.144312 28817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:23.146250 28746 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:37:23.146412 28746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "0bc1fa66524d4b13bb2f40b7880e0048"
format_stamp: "Formatted at 2026-08-07 08:37:21 on dist-test-slave-h3ft"
server_key: "1e042df00d3ccd8221306120b388f08e"
server_key_iv: "281fa25b04477e021297b31022f5852e"
server_key_version: "kuduclusterkey@0"
I20260807 08:37:23.147238 28746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:23.172168 28746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:23.177140 28746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:23.177484 28746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:23.178510 28746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:23.179966 28746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:23.180104 28746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 08:37:23.180274 28746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:23.180380 28746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:23.195178 28746 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:40937
I20260807 08:37:23.197111 28746 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:23.198195 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 28746
I20260807 08:37:23.198868 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.174.20:48797/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:36749
--local_ip_for_outbound_sockets=127.9.174.2
--tserver_master_addrs=127.9.174.62:44217
--webserver_port=32885
--webserver_interface=127.9.174.2
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:37413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.174.20:48797/kms with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:37:23.204561 28992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:40937 every 8 connection(s)
Aug 07 08:37:23 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091843, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:23.229387 28996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:23.202025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:44217 (local address 127.9.174.1:55043)
0807 08:37:23.205132 (+ 3107us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:23.205199 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:23.207468 (+ 2269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:23.210297 (+ 2829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:23.210311 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:23.210821 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:23.211810 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:23.211839 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:23.214306 (+ 2467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:23.214313 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:23.216157 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:23.216175 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:23.217686 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:23.217690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:23.217888 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:23.218793 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:23.218818 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:23.220978 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:23.224329 (+ 3351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:23.224341 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:23.224367 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:23.224980 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:23.225534 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:23.225537 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:23.225539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:23.225939 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:23.226890 (+ 951us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:23.226897 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:23.227324 (+ 427us) client_negotiation.cc:769] Sending connection context
0807 08:37:23.227545 (+ 221us) client_negotiation.cc:240] Negotiation successful
0807 08:37:23.228023 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2792,"thread_start_us":181,"threads_started":1}
I20260807 08:37:23.231498 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:23.203966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:55043 (local address 127.9.174.62:44217)
0807 08:37:23.205758 (+ 1792us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:23.205796 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:23.205860 (+ 64us) server_negotiation.cc:407] Connection header received
0807 08:37:23.208373 (+ 2513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:23.208392 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:23.209017 (+ 625us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:23.209525 (+ 508us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:23.212127 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:23.214072 (+ 1945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:23.216368 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:23.217193 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:23.217378 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:23.221298 (+ 3920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:23.221309 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:23.221331 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:23.221376 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:23.224041 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:23.225141 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:23.225150 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:23.225157 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:23.225418 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:23.226064 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:23.226067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:23.226069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:23.226679 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:23.226824 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:23.230110 (+ 3286us) server_negotiation.cc:299] Negotiation successful
0807 08:37:23.230463 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1542,"thread_start_us":146,"threads_started":1}
I20260807 08:37:23.235723 28993 heartbeater.cc:347] Connected to a master server at 127.9.174.62:44217
I20260807 08:37:23.236064 28993 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:23.236800 28993 heartbeater.cc:511] Master 127.9.174.62:44217 requested a full tablet report, sending...
W20260807 08:37:23.239902 28993 heartbeater.cc:644] Failed to heartbeat to 127.9.174.62:44217 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 08:37:23.240056 28993 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:23.240329 28993 heartbeater.cc:511] Master 127.9.174.62:44217 requested a full tablet report, sending...
I20260807 08:37:23.241575 28993 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:23.241781 28993 heartbeater.cc:511] Master 127.9.174.62:44217 requested a full tablet report, sending...
W20260807 08:37:23.242702 28993 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:37:23.366010 28995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:23.366381 28995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:23.366462 28995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:23.371807 28995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:23.371917 28995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:23.372014 28995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:37:23.378937 28995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:37413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.174.20:48797/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:36749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=32885
--tserver_master_addrs=127.9.174.62:44217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:23.380653 28995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:23.382015 28995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:23.395588 29008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:23.395864 29005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:23.395556 29006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:23.396096 28995 server_base.cc:1034] running on GCE node
I20260807 08:37:23.396616 28995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:23.397558 28995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:23.398753 28995 hybrid_clock.cc:648] HybridClock initialized: now 1786091843398722 us; error 52 us; skew 500 ppm
Aug 07 08:37:23 dist-test-slave-h3ft krb5kdc[27178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091843, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:23.402130 28995 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:37:23.403389 28995 webserver.cc:460] Webserver started at http://127.9.174.2:32885/ using document root <none> and password file <none>
I20260807 08:37:23.404212 28995 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:23.404382 28995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 08:37:23 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091843, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
I20260807 08:37:23.436965 28995 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.004s sys 0.000s
I20260807 08:37:23.440816 29023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:23.442598 28995 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:37:23.442761 28995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "b7db396ccda041ec8f51b3914ddcedb9"
format_stamp: "Formatted at 2026-08-07 08:37:21 on dist-test-slave-h3ft"
server_key: "790c1855ea7ec91895c5c2c8187d487d"
server_key_iv: "b35cea9224519f374770ba22ef59193a"
server_key_version: "kuduclusterkey@0"
I20260807 08:37:23.443532 28995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:23.475174 28995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:23.479306 28995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:23.479574 28995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:23.480438 28995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:23.481714 28995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:23.481791 28995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:23.481900 28995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:23.481986 28995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:23.496241 28995 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:36749
I20260807 08:37:23.496410 29192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:36749 every 8 connection(s)
I20260807 08:37:23.497823 28995 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:23.502686 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 28995
I20260807 08:37:23.503358 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.174.20:48797/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:38337
--local_ip_for_outbound_sockets=127.9.174.3
--tserver_master_addrs=127.9.174.62:44217
--webserver_port=39969
--webserver_interface=127.9.174.3
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:37413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.174.20:48797/kms with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:37:23 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091843, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:23.523406 29196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:23.500924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:44217 (local address 127.9.174.2:55625)
0807 08:37:23.501517 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:23.501579 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:23.502951 (+ 1372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:23.503901 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:23.503912 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:23.504363 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:23.505201 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:23.505226 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:23.508382 (+ 3156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:23.508386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:23.510014 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:23.510030 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:23.512379 (+ 2349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:23.512383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:23.512663 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:23.513799 (+ 1136us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:23.513827 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:23.516055 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:23.518629 (+ 2574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:23.518637 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:23.518664 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:23.519199 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:23.519605 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:23.519608 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:23.519610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:23.519855 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:23.520523 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:23.520532 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:23.521062 (+ 530us) client_negotiation.cc:769] Sending connection context
0807 08:37:23.521350 (+ 288us) client_negotiation.cc:240] Negotiation successful
0807 08:37:23.521848 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":178,"threads_started":1}
I20260807 08:37:23.523674 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:23.500356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:55625 (local address 127.9.174.62:44217)
0807 08:37:23.500543 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:23.500549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:23.501723 (+ 1174us) server_negotiation.cc:407] Connection header received
0807 08:37:23.503311 (+ 1588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:23.503316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:23.503398 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:23.503509 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:23.506789 (+ 3280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:23.507789 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:23.511555 (+ 3766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:23.511993 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:23.512119 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:23.516352 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:23.516357 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:23.516360 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:23.516406 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:23.518448 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:23.519349 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:23.519354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:23.519356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:23.519476 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:23.520006 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:23.520009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:23.520011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:23.520334 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:23.520449 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:23.523202 (+ 2753us) server_negotiation.cc:299] Negotiation successful
0807 08:37:23.523403 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 08:37:23.525704 29193 heartbeater.cc:347] Connected to a master server at 127.9.174.62:44217
I20260807 08:37:23.526060 29193 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:23.526875 29193 heartbeater.cc:511] Master 127.9.174.62:44217 requested a full tablet report, sending...
W20260807 08:37:23.529464 29193 heartbeater.cc:644] Failed to heartbeat to 127.9.174.62:44217 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 08:37:23.529620 29193 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:23.529871 29193 heartbeater.cc:511] Master 127.9.174.62:44217 requested a full tablet report, sending...
I20260807 08:37:23.531076 29193 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:23.531279 29193 heartbeater.cc:511] Master 127.9.174.62:44217 requested a full tablet report, sending...
W20260807 08:37:23.532258 29193 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:37:23.680060 29200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:23.680670 29200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:23.680804 29200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:23.687196 29200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:23.687325 29200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:23.687448 29200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:37:23.694257 29200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:37413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.174.20:48797/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:38337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=39969
--tserver_master_addrs=127.9.174.62:44217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:23.696080 29200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:23.697486 29200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:23.711814 29209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:23.712636 29215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:23.712742 29210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:23.713066 29200 server_base.cc:1034] running on GCE node
I20260807 08:37:23.713629 29200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:23.714566 29200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:23.715759 29200 hybrid_clock.cc:648] HybridClock initialized: now 1786091843715721 us; error 55 us; skew 500 ppm
Aug 07 08:37:23 dist-test-slave-h3ft krb5kdc[27178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091843, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:23.719879 29200 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:37:23.721421 29200 webserver.cc:460] Webserver started at http://127.9.174.3:39969/ using document root <none> and password file <none>
I20260807 08:37:23.722373 29200 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:23.722563 29200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 08:37:23 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091843, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
I20260807 08:37:23.758483 29200 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 08:37:23.762899 29229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:23.766786 29200 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.002s
I20260807 08:37:23.767102 29200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "304a34e4aa76450a9e542e5a3c184780"
format_stamp: "Formatted at 2026-08-07 08:37:22 on dist-test-slave-h3ft"
server_key: "5f06b3da7c37c60e997fa1ac29d259cf"
server_key_iv: "1a0659eae8e0db1847623ab9d44fdf3d"
server_key_version: "kuduclusterkey@0"
I20260807 08:37:23.768301 29200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:23.810940 29200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:23.815243 29200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:23.815459 29200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:23.816305 29200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:23.817731 29200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:23.817853 29200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:23.817950 29200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:23.817997 29200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:23.831243 29200 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:38337
I20260807 08:37:23.832738 29200 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:37:23.833019 29408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:38337 every 8 connection(s)
I20260807 08:37:23.837936 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 29200
Aug 07 08:37:23 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091843, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:23.851814 29416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:23.838889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44958 (local address 127.9.174.62:44217)
0807 08:37:23.839303 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:23.839312 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:23.839671 (+ 359us) server_negotiation.cc:407] Connection header received
0807 08:37:23.839786 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:23.839790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:23.839853 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:23.839982 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:23.841010 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:23.842019 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:23.843354 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:23.843831 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:23.843963 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:23.846978 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:23.846982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:23.846986 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:23.847045 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:23.849274 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:23.850205 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:23.850209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:23.850211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:23.850323 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:23.850669 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:23.850674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:23.850676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:23.850951 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:23.851240 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:23.851383 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 08:37:23.851563 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":330,"thread_start_us":194,"threads_started":1}
I20260807 08:37:23.852638 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:23.835349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:33365 (local address 127.9.174.62:44217)
0807 08:37:23.835588 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:23.835596 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:23.836891 (+ 1295us) server_negotiation.cc:407] Connection header received
0807 08:37:23.838529 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:23.838533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:23.838600 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:23.838702 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:23.841031 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:23.841653 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:23.843479 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:23.843776 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:23.843899 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:23.847474 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:23.847478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:23.847480 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:23.847533 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:23.849450 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:23.850218 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:23.850222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:23.850224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:23.850345 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:23.850832 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:23.850835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:23.850837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:23.851086 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:23.851180 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:23.852310 (+ 1130us) server_negotiation.cc:299] Negotiation successful
0807 08:37:23.852449 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":91}
I20260807 08:37:23.854856 29414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:23.835948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:44217 (local address 127.9.174.3:33365)
0807 08:37:23.836652 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:23.836728 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:23.838187 (+ 1459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:23.839511 (+ 1324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:23.839523 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:23.839978 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:23.840880 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:23.840906 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:23.841791 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:23.841795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:23.843349 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:23.843360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:23.843959 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:23.843964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:23.844200 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:23.845203 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:23.845232 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:23.847264 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:23.849599 (+ 2335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:23.849607 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:23.849631 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:23.850070 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:23.850477 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:23.850482 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:23.850487 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:23.850723 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:23.851241 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:23.851251 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:23.851726 (+ 475us) client_negotiation.cc:769] Sending connection context
0807 08:37:23.853062 (+ 1336us) client_negotiation.cc:240] Negotiation successful
0807 08:37:23.853578 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":163,"threads_started":1}
I20260807 08:37:23.856261 29410 heartbeater.cc:347] Connected to a master server at 127.9.174.62:44217
I20260807 08:37:23.856587 29410 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:23.857334 29410 heartbeater.cc:511] Master 127.9.174.62:44217 requested a full tablet report, sending...
I20260807 08:37:23.859181 28692 ts_manager.cc:194] Registered new tserver with Master: 304a34e4aa76450a9e542e5a3c184780 (127.9.174.3:38337)
I20260807 08:37:23.860687 28692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:33365
Aug 07 08:37:24 dist-test-slave-h3ft krb5kdc[27178](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786091844, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:24.243053 28993 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:24.243459 28993 heartbeater.cc:511] Master 127.9.174.62:44217 requested a full tablet report, sending...
I20260807 08:37:24.244508 28692 ts_manager.cc:194] Registered new tserver with Master: 0bc1fa66524d4b13bb2f40b7880e0048 (127.9.174.1:40937)
I20260807 08:37:24.245402 28692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:55043
I20260807 08:37:24.532626 29193 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:24.533075 29193 heartbeater.cc:511] Master 127.9.174.62:44217 requested a full tablet report, sending...
I20260807 08:37:24.534198 28692 ts_manager.cc:194] Registered new tserver with Master: b7db396ccda041ec8f51b3914ddcedb9 (127.9.174.2:36749)
I20260807 08:37:24.535192 28692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:55625
I20260807 08:37:24.543807 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:37:24.561558 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:24.545901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44966 (local address 127.9.174.62:44217)
0807 08:37:24.546268 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:24.546273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:24.546374 (+ 101us) server_negotiation.cc:407] Connection header received
0807 08:37:24.546568 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:24.546572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:24.546637 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:24.546739 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:24.547706 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.548583 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.549916 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.550375 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.550562 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:24.557144 (+ 6582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:24.557149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:24.557152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:24.557193 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:24.559323 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:24.559878 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:24.559882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:24.559884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:24.560003 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:24.560347 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:24.560351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:24.560353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:24.560643 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:24.560805 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:24.561132 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 08:37:24.561307 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":143,"threads_started":1}
I20260807 08:37:24.566874 28692 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:24.572868 28692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:24.612341 29444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:24.593030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:40937 (local address 127.0.0.1:45272)
0807 08:37:24.599713 (+ 6683us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:24.599811 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:24.599995 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:24.601638 (+ 1643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:24.601646 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:24.601680 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:24.601903 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:24.601914 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.603777 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.603781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:24.610162 (+ 6381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:24.610178 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.611554 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.611559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:24.611712 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:24.611773 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 08:37:24.611892 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 08:37:24.612091 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6369,"thread_start_us":159,"threads_started":1}
I20260807 08:37:24.614193 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:24.593254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45272 (local address 127.9.174.1:40937)
0807 08:37:24.600911 (+ 7657us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:24.600922 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:24.600958 (+ 36us) server_negotiation.cc:407] Connection header received
0807 08:37:24.601067 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:24.601073 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:24.601387 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:24.602113 (+ 726us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:24.602322 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.603620 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.610387 (+ 6767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.611370 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.613552 (+ 2182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:24.613785 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:24.613861 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 08:37:24.613998 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7499,"thread_start_us":269,"threads_started":1}
I20260807 08:37:24.619788 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:24.595643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:38337 (local address 127.0.0.1:49278)
0807 08:37:24.600470 (+ 4827us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:24.600483 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:24.600570 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:24.603948 (+ 3378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:24.603951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:24.603972 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:24.604088 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:24.604095 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.608678 (+ 4583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.608682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:24.610069 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:24.610083 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.619261 (+ 9178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.619266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:24.619391 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:24.619442 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 08:37:24.619508 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:37:24.619609 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4773,"thread_start_us":97,"threads_started":1}
I20260807 08:37:24.620733 29445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:24.595895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49278 (local address 127.9.174.3:38337)
0807 08:37:24.603342 (+ 7447us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:24.603351 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:24.603383 (+ 32us) server_negotiation.cc:407] Connection header received
0807 08:37:24.603488 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:24.603493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:24.603740 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:24.604283 (+ 543us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:24.604476 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.605668 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.610264 (+ 4596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.611363 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.620085 (+ 8722us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:24.620330 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:24.620405 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 08:37:24.620555 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7291,"thread_start_us":130,"threads_started":1}
I20260807 08:37:24.620546 28887 tablet_service.cc:1467] Processing CreateTablet for tablet 2add394a020944afb3686211b508c2af (DEFAULT_TABLE table=test-table [id=2cb7de81687b49ebbeb84bcc10888943]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:24.622068 28887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2add394a020944afb3686211b508c2af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:24.622648 29441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:24.594870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:36749 (local address 127.0.0.1:48620)
0807 08:37:24.600217 (+ 5347us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:24.600229 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:24.600294 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:24.606629 (+ 6335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:24.606633 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:24.606656 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:24.606777 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:24.606785 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.619146 (+ 12361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.619150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:24.620383 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:24.620396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.622199 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.622202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:24.622312 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:24.622340 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 08:37:24.622394 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 08:37:24.622474 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5290,"thread_start_us":130,"threads_started":1}
I20260807 08:37:24.623592 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:24.595268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48620 (local address 127.9.174.2:36749)
0807 08:37:24.606051 (+ 10783us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:24.606060 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:24.606093 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:37:24.606190 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:24.606195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:24.606439 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:24.606970 (+ 531us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:24.607236 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.608477 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.620714 (+ 12237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.622023 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.622957 (+ 934us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:24.623184 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:24.623246 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 08:37:24.623359 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10623,"thread_start_us":156,"threads_started":1}
I20260807 08:37:24.624321 29297 tablet_service.cc:1467] Processing CreateTablet for tablet 2add394a020944afb3686211b508c2af (DEFAULT_TABLE table=test-table [id=2cb7de81687b49ebbeb84bcc10888943]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:24.625811 29297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2add394a020944afb3686211b508c2af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:24.629734 29100 tablet_service.cc:1467] Processing CreateTablet for tablet 2add394a020944afb3686211b508c2af (DEFAULT_TABLE table=test-table [id=2cb7de81687b49ebbeb84bcc10888943]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:24.631138 29100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2add394a020944afb3686211b508c2af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:24.633561 29446 tablet_bootstrap.cc:492] T 2add394a020944afb3686211b508c2af P 0bc1fa66524d4b13bb2f40b7880e0048: Bootstrap starting.
I20260807 08:37:24.636765 29446 tablet_bootstrap.cc:654] T 2add394a020944afb3686211b508c2af P 0bc1fa66524d4b13bb2f40b7880e0048: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:24.637873 29446 log.cc:824] T 2add394a020944afb3686211b508c2af P 0bc1fa66524d4b13bb2f40b7880e0048: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:24.640925 29450 tablet_bootstrap.cc:492] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9: Bootstrap starting.
I20260807 08:37:24.643970 29446 tablet_bootstrap.cc:492] T 2add394a020944afb3686211b508c2af P 0bc1fa66524d4b13bb2f40b7880e0048: No bootstrap required, opened a new log
I20260807 08:37:24.644840 29446 ts_tablet_manager.cc:1397] T 2add394a020944afb3686211b508c2af P 0bc1fa66524d4b13bb2f40b7880e0048: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.004s
I20260807 08:37:24.648046 29451 tablet_bootstrap.cc:492] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780: Bootstrap starting.
I20260807 08:37:24.649859 29446 raft_consensus.cc:348] T 2add394a020944afb3686211b508c2af P 0bc1fa66524d4b13bb2f40b7880e0048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1fa66524d4b13bb2f40b7880e0048" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 40937 } } peers { permanent_uuid: "b7db396ccda041ec8f51b3914ddcedb9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36749 } } peers { permanent_uuid: "304a34e4aa76450a9e542e5a3c184780" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38337 } }
I20260807 08:37:24.650413 29446 raft_consensus.cc:374] T 2add394a020944afb3686211b508c2af P 0bc1fa66524d4b13bb2f40b7880e0048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:24.650547 29446 raft_consensus.cc:729] T 2add394a020944afb3686211b508c2af P 0bc1fa66524d4b13bb2f40b7880e0048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bc1fa66524d4b13bb2f40b7880e0048, State: Initialized, Role: FOLLOWER
I20260807 08:37:24.650573 29450 tablet_bootstrap.cc:654] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:24.651101 29451 tablet_bootstrap.cc:654] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:24.651518 29446 consensus_queue.cc:260] T 2add394a020944afb3686211b508c2af P 0bc1fa66524d4b13bb2f40b7880e0048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1fa66524d4b13bb2f40b7880e0048" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 40937 } } peers { permanent_uuid: "b7db396ccda041ec8f51b3914ddcedb9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36749 } } peers { permanent_uuid: "304a34e4aa76450a9e542e5a3c184780" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38337 } }
I20260807 08:37:24.651695 29450 log.cc:824] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:24.651968 29451 log.cc:824] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:24.653108 29446 ts_tablet_manager.cc:1428] T 2add394a020944afb3686211b508c2af P 0bc1fa66524d4b13bb2f40b7880e0048: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 08:37:24.653679 28993 heartbeater.cc:503] Master 127.9.174.62:44217 was elected leader, sending a full tablet report...
I20260807 08:37:24.655189 29451 tablet_bootstrap.cc:492] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780: No bootstrap required, opened a new log
I20260807 08:37:24.655435 29451 ts_tablet_manager.cc:1397] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 08:37:24.661473 29451 raft_consensus.cc:348] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1fa66524d4b13bb2f40b7880e0048" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 40937 } } peers { permanent_uuid: "b7db396ccda041ec8f51b3914ddcedb9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36749 } } peers { permanent_uuid: "304a34e4aa76450a9e542e5a3c184780" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38337 } }
I20260807 08:37:24.661808 29451 raft_consensus.cc:374] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:24.662533 29451 raft_consensus.cc:729] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 304a34e4aa76450a9e542e5a3c184780, State: Initialized, Role: FOLLOWER
I20260807 08:37:24.663249 29451 consensus_queue.cc:260] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1fa66524d4b13bb2f40b7880e0048" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 40937 } } peers { permanent_uuid: "b7db396ccda041ec8f51b3914ddcedb9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36749 } } peers { permanent_uuid: "304a34e4aa76450a9e542e5a3c184780" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38337 } }
I20260807 08:37:24.665759 29410 heartbeater.cc:503] Master 127.9.174.62:44217 was elected leader, sending a full tablet report...
I20260807 08:37:24.667573 29451 ts_tablet_manager.cc:1428] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
I20260807 08:37:24.670876 29459 raft_consensus.cc:482] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:24.672405 29459 raft_consensus.cc:504] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1fa66524d4b13bb2f40b7880e0048" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 40937 } } peers { permanent_uuid: "b7db396ccda041ec8f51b3914ddcedb9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36749 } } peers { permanent_uuid: "304a34e4aa76450a9e542e5a3c184780" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38337 } }
I20260807 08:37:24.674794 29459 leader_election.cc:288] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bc1fa66524d4b13bb2f40b7880e0048 (127.9.174.1:40937), b7db396ccda041ec8f51b3914ddcedb9 (127.9.174.2:36749)
I20260807 08:37:24.677064 29450 tablet_bootstrap.cc:492] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9: No bootstrap required, opened a new log
I20260807 08:37:24.677522 29450 ts_tablet_manager.cc:1397] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9: Time spent bootstrapping tablet: real 0.037s user 0.008s sys 0.000s
I20260807 08:37:24.682845 29450 raft_consensus.cc:348] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1fa66524d4b13bb2f40b7880e0048" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 40937 } } peers { permanent_uuid: "b7db396ccda041ec8f51b3914ddcedb9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36749 } } peers { permanent_uuid: "304a34e4aa76450a9e542e5a3c184780" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38337 } }
I20260807 08:37:24.683470 29450 raft_consensus.cc:374] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:24.683597 29450 raft_consensus.cc:729] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7db396ccda041ec8f51b3914ddcedb9, State: Initialized, Role: FOLLOWER
I20260807 08:37:24.684449 29450 consensus_queue.cc:260] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1fa66524d4b13bb2f40b7880e0048" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 40937 } } peers { permanent_uuid: "b7db396ccda041ec8f51b3914ddcedb9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36749 } } peers { permanent_uuid: "304a34e4aa76450a9e542e5a3c184780" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38337 } }
I20260807 08:37:24.684902 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:24.675842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:47371 (local address 127.9.174.2:36749)
0807 08:37:24.676015 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:24.676019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:24.679445 (+ 3426us) server_negotiation.cc:407] Connection header received
0807 08:37:24.679701 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:24.679706 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:24.679770 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:24.679844 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:24.680834 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.681675 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.683414 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.684050 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.684139 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:24.684232 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:24.684639 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 08:37:24.684741 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 08:37:24.685704 29463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:24.675317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:36749 (local address 127.9.174.3:47371)
0807 08:37:24.679305 (+ 3988us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:24.679323 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:24.679557 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:24.680462 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:24.680468 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:24.680494 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:24.680685 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:24.680692 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.681854 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.681858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:24.683242 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:24.683257 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.684341 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.684345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:24.684477 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:24.684507 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 08:37:24.685391 (+ 884us) client_negotiation.cc:240] Negotiation successful
0807 08:37:24.685494 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3890,"thread_start_us":113,"threads_started":1}
I20260807 08:37:24.686424 29450 ts_tablet_manager.cc:1428] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260807 08:37:24.686777 29105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2add394a020944afb3686211b508c2af" candidate_uuid: "304a34e4aa76450a9e542e5a3c184780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7db396ccda041ec8f51b3914ddcedb9" is_pre_election: true
I20260807 08:37:24.687033 29193 heartbeater.cc:503] Master 127.9.174.62:44217 was elected leader, sending a full tablet report...
I20260807 08:37:24.687623 29105 raft_consensus.cc:2454] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 304a34e4aa76450a9e542e5a3c184780 in term 0.
I20260807 08:37:24.688433 29230 leader_election.cc:302] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 304a34e4aa76450a9e542e5a3c184780, b7db396ccda041ec8f51b3914ddcedb9; no voters:
I20260807 08:37:24.689038 29459 raft_consensus.cc:2781] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:24.689200 29459 raft_consensus.cc:482] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:24.689260 29459 raft_consensus.cc:3037] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:24.690992 29459 raft_consensus.cc:504] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1fa66524d4b13bb2f40b7880e0048" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 40937 } } peers { permanent_uuid: "b7db396ccda041ec8f51b3914ddcedb9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36749 } } peers { permanent_uuid: "304a34e4aa76450a9e542e5a3c184780" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38337 } }
I20260807 08:37:24.691149 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:24.674480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:59669 (local address 127.9.174.1:40937)
0807 08:37:24.674657 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:24.674663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:24.680240 (+ 5577us) server_negotiation.cc:407] Connection header received
0807 08:37:24.683368 (+ 3128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:24.683373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:24.683449 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:24.683506 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:24.684018 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.684901 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.687251 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.687983 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.688080 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:24.688174 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:24.690786 (+ 2612us) server_negotiation.cc:299] Negotiation successful
0807 08:37:24.690946 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 08:37:24.691625 29459 leader_election.cc:288] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [CANDIDATE]: Term 1 election: Requested vote from peers 0bc1fa66524d4b13bb2f40b7880e0048 (127.9.174.1:40937), b7db396ccda041ec8f51b3914ddcedb9 (127.9.174.2:36749)
I20260807 08:37:24.691648 29465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:24.674338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:40937 (local address 127.9.174.3:59669)
0807 08:37:24.677620 (+ 3282us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:24.680127 (+ 2507us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:24.683148 (+ 3021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:24.683669 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:24.683673 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:24.683692 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:24.683865 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:24.683871 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.685062 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.685065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:24.686762 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:24.686776 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.688297 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.688301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:24.688427 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:24.688456 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 08:37:24.691355 (+ 2899us) client_negotiation.cc:240] Negotiation successful
0807 08:37:24.691475 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3190,"thread_start_us":180,"threads_started":1}
I20260807 08:37:24.692559 28895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2add394a020944afb3686211b508c2af" candidate_uuid: "304a34e4aa76450a9e542e5a3c184780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bc1fa66524d4b13bb2f40b7880e0048" is_pre_election: true
I20260807 08:37:24.692600 28917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2add394a020944afb3686211b508c2af" candidate_uuid: "304a34e4aa76450a9e542e5a3c184780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bc1fa66524d4b13bb2f40b7880e0048"
I20260807 08:37:24.693018 28895 raft_consensus.cc:2454] T 2add394a020944afb3686211b508c2af P 0bc1fa66524d4b13bb2f40b7880e0048 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 304a34e4aa76450a9e542e5a3c184780 in term 0.
I20260807 08:37:24.695199 29105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2add394a020944afb3686211b508c2af" candidate_uuid: "304a34e4aa76450a9e542e5a3c184780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7db396ccda041ec8f51b3914ddcedb9"
I20260807 08:37:24.695364 29105 raft_consensus.cc:3037] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:24.697192 29105 raft_consensus.cc:2454] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 304a34e4aa76450a9e542e5a3c184780 in term 1.
I20260807 08:37:24.697741 29230 leader_election.cc:302] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 304a34e4aa76450a9e542e5a3c184780, b7db396ccda041ec8f51b3914ddcedb9; no voters: 0bc1fa66524d4b13bb2f40b7880e0048
I20260807 08:37:24.698015 29459 raft_consensus.cc:2781] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:24.698344 29459 raft_consensus.cc:686] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [term 1 LEADER]: Becoming Leader. State: Replica: 304a34e4aa76450a9e542e5a3c184780, State: Running, Role: LEADER
I20260807 08:37:24.699436 29459 consensus_queue.cc:237] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1fa66524d4b13bb2f40b7880e0048" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 40937 } } peers { permanent_uuid: "b7db396ccda041ec8f51b3914ddcedb9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36749 } } peers { permanent_uuid: "304a34e4aa76450a9e542e5a3c184780" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38337 } }
W20260807 08:37:24.700186 28994 tablet.cc:2369] T 2add394a020944afb3686211b508c2af P 0bc1fa66524d4b13bb2f40b7880e0048: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:37:24.704038 28692 catalog_manager.cc:5495] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 reported cstate change: term changed from 0 to 1, leader changed from <none> to 304a34e4aa76450a9e542e5a3c184780 (127.9.174.3). New cstate: current_term: 1 leader_uuid: "304a34e4aa76450a9e542e5a3c184780" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1fa66524d4b13bb2f40b7880e0048" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 40937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7db396ccda041ec8f51b3914ddcedb9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "304a34e4aa76450a9e542e5a3c184780" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38337 } health_report { overall_health: HEALTHY } } }
Aug 07 08:37:24 dist-test-slave-h3ft krb5kdc[27178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091844, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for HTTP/127.9.174.20@KRBTEST.COM
I20260807 08:37:24.734030 29445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:24.727362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49288 (local address 127.9.174.3:38337)
0807 08:37:24.727549 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:24.727555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:24.727835 (+ 280us) server_negotiation.cc:407] Connection header received
0807 08:37:24.727985 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:24.727990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:24.728091 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:24.728238 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:24.728723 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.729682 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.731257 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:24.731665 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:24.731776 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:24.732344 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:24.733177 (+ 833us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:24.733319 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:24.733730 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 08:37:24.733817 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 08:37:24.748543 28917 raft_consensus.cc:3037] T 2add394a020944afb3686211b508c2af P 0bc1fa66524d4b13bb2f40b7880e0048 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:24.748726 29105 raft_consensus.cc:1261] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9 [term 1 FOLLOWER]: Refusing update from remote peer 304a34e4aa76450a9e542e5a3c184780: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:24.750705 28917 raft_consensus.cc:1261] T 2add394a020944afb3686211b508c2af P 0bc1fa66524d4b13bb2f40b7880e0048 [term 1 FOLLOWER]: Refusing update from remote peer 304a34e4aa76450a9e542e5a3c184780: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 08:37:24.750734 29194 tablet.cc:2369] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:37:24.751986 29459 consensus_queue.cc:1035] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bc1fa66524d4b13bb2f40b7880e0048" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 40937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:24.753512 29459 consensus_queue.cc:1035] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7db396ccda041ec8f51b3914ddcedb9" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 36749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:24.786688 29474 mvcc.cc:205] Tried to move back new op lower bound from 7315832196078796800 to 7315832195893239808. Current Snapshot: MvccSnapshot[applied={T|T < 7315832196078796800}]
I20260807 08:37:24.792645 29472 mvcc.cc:205] Tried to move back new op lower bound from 7315832196078796800 to 7315832195893239808. Current Snapshot: MvccSnapshot[applied={T|T < 7315832196078796800}]
I20260807 08:37:24.807137 28692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:24.807408 28692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:24.811213 28692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 219b893e21ce43c48f4d03c10305fa12: Sending DeleteTablet for 3 replicas of tablet 2add394a020944afb3686211b508c2af
I20260807 08:37:24.813469 28887 tablet_service.cc:1514] Processing DeleteTablet for tablet 2add394a020944afb3686211b508c2af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:24 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:45272
I20260807 08:37:24.813457 29297 tablet_service.cc:1514] Processing DeleteTablet for tablet 2add394a020944afb3686211b508c2af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:24 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:49278
I20260807 08:37:24.814221 29100 tablet_service.cc:1514] Processing DeleteTablet for tablet 2add394a020944afb3686211b508c2af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:24 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:48620
I20260807 08:37:24.815863 29486 tablet_replica.cc:330] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780: stopping tablet replica
I20260807 08:37:24.816308 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 28746
I20260807 08:37:24.816350 29486 raft_consensus.cc:2229] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:37:24.817775 29486 raft_consensus.cc:2258] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:24.820158 29486 ts_tablet_manager.cc:1905] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:24.824288 29486 ts_tablet_manager.cc:1918] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:24.824440 29486 log.cc:1196] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/2add394a020944afb3686211b508c2af
I20260807 08:37:24.825016 29486 ts_tablet_manager.cc:1939] T 2add394a020944afb3686211b508c2af P 304a34e4aa76450a9e542e5a3c184780: Deleting consensus metadata
I20260807 08:37:24.828042 28669 catalog_manager.cc:4841] TS 304a34e4aa76450a9e542e5a3c184780 (127.9.174.3:38337): tablet 2add394a020944afb3686211b508c2af (table test-table [id=2cb7de81687b49ebbeb84bcc10888943]) successfully deleted
I20260807 08:37:24.851188 29489 tablet_replica.cc:330] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9: stopping tablet replica
I20260807 08:37:24.852016 29489 raft_consensus.cc:2229] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:24.859978 29489 raft_consensus.cc:2258] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:24.862818 29489 ts_tablet_manager.cc:1905] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:24.872610 29489 ts_tablet_manager.cc:1918] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:24.874868 29489 log.cc:1196] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/2add394a020944afb3686211b508c2af
I20260807 08:37:24.875546 29489 ts_tablet_manager.cc:1939] T 2add394a020944afb3686211b508c2af P b7db396ccda041ec8f51b3914ddcedb9: Deleting consensus metadata
W20260807 08:37:24.876075 28674 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:40937 (error 108)
I20260807 08:37:24.879521 28669 catalog_manager.cc:4841] TS b7db396ccda041ec8f51b3914ddcedb9 (127.9.174.2:36749): tablet 2add394a020944afb3686211b508c2af (table test-table [id=2cb7de81687b49ebbeb84bcc10888943]) successfully deleted
I20260807 08:37:24.880138 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 28995
I20260807 08:37:24.894903 29441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:24.894388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:40937 (local address 127.0.0.1:45276)
0807 08:37:24.894679 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:24.894783 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:40937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174}
W20260807 08:37:24.895458 28674 catalog_manager.cc:4584] TS 0bc1fa66524d4b13bb2f40b7880e0048 (127.9.174.1:40937): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2add394a020944afb3686211b508c2af: Network error: Client connection negotiation failed: client connection to 127.9.174.1:40937: connect: Connection refused (error 111)
I20260807 08:37:24.905342 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 29200
I20260807 08:37:24.942792 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 28642
I20260807 08:37:24.957758 9912 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 08:37:25.308315 9912 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 08:37:25.308483 9912 mini_ranger.cc:66] Stopping Ranger...
I20260807 08:37:25.664985 9912 mini_ranger.cc:68] Stopped Ranger
2026-08-07 08:37:25.665 UTC [27222] LOG: received smart shutdown request
2026-08-07 08:37:25.670 UTC [27222] LOG: background worker "logical replication launcher" (PID 27237) exited with exit code 1
2026-08-07 08:37:25.670 UTC [27228] LOG: shutting down
2026-08-07 08:37:25.857 UTC [27222] LOG: database system is shut down
2026-08-07T08:37:25Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (57406 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:26 dist-test-slave-h3ft krb5kdc[29504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:26 dist-test-slave-h3ft krb5kdc[29504](info): set up 2 sockets
Aug 07 08:37:26 dist-test-slave-h3ft krb5kdc[29504](info): commencing operation
krb5kdc: starting...
W20260807 08:37:27.259707 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:37:27 dist-test-slave-h3ft krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:37:27.415364 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:33937
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:39749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:33937,127.9.174.61:38385,127.9.174.60:32949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:27.549819 29522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:27.550271 29522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:27.550359 29522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:27.554863 29522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:27.555027 29522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:27.555068 29522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:27.555114 29522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:27.555143 29522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:27.561343 29522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:39749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:33937,127.9.174.61:38385,127.9.174.60:32949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:33937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:27.562816 29522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:27.564234 29522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:27.575135 29530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:27.575135 29531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:27.575135 29533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:27.575268 29522 server_base.cc:1034] running on GCE node
I20260807 08:37:27.576295 29522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:27.577786 29522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:27.579039 29522 hybrid_clock.cc:648] HybridClock initialized: now 1786091847578994 us; error 40 us; skew 500 ppm
Aug 07 08:37:27 dist-test-slave-h3ft krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:27.582546 29522 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:37:27.583832 29522 webserver.cc:460] Webserver started at http://127.9.174.62:38385/ using document root <none> and password file <none>
I20260807 08:37:27.584618 29522 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:27.584748 29522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:27.585047 29522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:27.587713 29522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "66605a02d4fb4380a9b2168b5e4c9b3d"
format_stamp: "Formatted at 2026-08-07 08:37:27 on dist-test-slave-h3ft"
server_key: "829db5e3a959f593912bbcba344102df"
server_key_iv: "bbda005242574acf409f65ddb8dc1d4b"
server_key_version: "encryptionkey@0"
I20260807 08:37:27.588411 29522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "66605a02d4fb4380a9b2168b5e4c9b3d"
format_stamp: "Formatted at 2026-08-07 08:37:27 on dist-test-slave-h3ft"
server_key: "829db5e3a959f593912bbcba344102df"
server_key_iv: "bbda005242574acf409f65ddb8dc1d4b"
server_key_version: "encryptionkey@0"
I20260807 08:37:27.593334 29522 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 08:37:27.596531 29544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:27.598112 29522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 08:37:27.598306 29522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "66605a02d4fb4380a9b2168b5e4c9b3d"
format_stamp: "Formatted at 2026-08-07 08:37:27 on dist-test-slave-h3ft"
server_key: "829db5e3a959f593912bbcba344102df"
server_key_iv: "bbda005242574acf409f65ddb8dc1d4b"
server_key_version: "encryptionkey@0"
I20260807 08:37:27.598464 29522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:27.638381 29522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:27.642828 29522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:27.643152 29522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:27.652510 29522 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:33937
I20260807 08:37:27.652539 29637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:33937 every 8 connection(s)
I20260807 08:37:27.653769 29522 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:27.656354 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 29522
I20260807 08:37:27.656483 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:37:27.656838 9912 external_mini_cluster.cc:1428] Setting key a8b79fc98373dfb9bb0196901e6b28f5
I20260807 08:37:27.657501 29638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:27.662348 29638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.174.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.61@KRBTEST.COM" created.
Aug 07 08:37:27 dist-test-slave-h3ft krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:27.681301 29643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.665389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57862 (local address 127.9.174.62:33937)
0807 08:37:27.665932 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:27.665956 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:27.666463 (+ 507us) server_negotiation.cc:407] Connection header received
0807 08:37:27.667997 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:27.668007 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:27.668261 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:27.668452 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:27.669563 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.670732 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.672027 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.672460 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.672681 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:27.675592 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:27.675599 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:27.675612 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:27.675635 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:27.677460 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:27.678000 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:27.678006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:27.678008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:27.678130 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:27.678483 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:27.678488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:27.678495 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:27.678969 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:27.679190 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:27.679617 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 08:37:27.679957 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":350,"thread_start_us":163,"threads_started":1}
I20260807 08:37:27.681298 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.665401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:33937 (local address 127.0.0.1:57862)
0807 08:37:27.666243 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:27.666314 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:27.667576 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:27.668517 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:27.668523 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:27.669050 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:27.669360 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:27.669376 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.670925 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.670929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:27.671883 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:27.671896 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.672666 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.672670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:27.672788 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:27.673595 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:27.673622 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:27.675419 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:27.677594 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:27.677605 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:27.677618 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:27.677870 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:27.678231 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:27.678236 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:27.678238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:27.678380 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:27.679149 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:27.679157 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:27.679329 (+ 172us) client_negotiation.cc:769] Sending connection context
0807 08:37:27.679538 (+ 209us) client_negotiation.cc:240] Negotiation successful
0807 08:37:27.680009 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"mutex_wait_us":122,"thread_start_us":371,"threads_started":1}
I20260807 08:37:27.684635 29638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:37:27.685638 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.685281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:38385 (local address 127.0.0.1:36448)
0807 08:37:27.685454 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:27.685545 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 08:37:27.685945 29549 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.174.61:38385: connect: Connection refused (error 111)
I20260807 08:37:27.687148 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.686830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:38385 (local address 127.0.0.1:36458)
0807 08:37:27.686958 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:27.687059 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 08:37:27.687342 29638 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.61:38385: Network error: Client connection negotiation failed: client connection to 127.9.174.61:38385: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.61.keytab.
Entry for principal kudu/127.9.174.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.61.keytab.
I20260807 08:37:27.690871 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.61:38385
--webserver_interface=127.9.174.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.61.keytab
--principal=kudu/127.9.174.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:39749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:33937,127.9.174.61:38385,127.9.174.60:32949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:37:27.736708 29638 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } attempt: 1
I20260807 08:37:27.738075 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.737623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:38385 (local address 127.0.0.1:36464)
0807 08:37:27.737853 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:27.737946 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 08:37:27.739114 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.738796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:38385 (local address 127.0.0.1:36474)
0807 08:37:27.738936 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:27.738996 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 08:37:27.739295 29638 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.61:38385: Network error: Client connection negotiation failed: client connection to 127.9.174.61:38385: connect: Connection refused (error 111)
I20260807 08:37:27.807518 29638 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } attempt: 2
I20260807 08:37:27.808756 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.808288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:38385 (local address 127.0.0.1:36486)
0807 08:37:27.808508 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:27.808619 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 08:37:27.809742 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.809503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:38385 (local address 127.0.0.1:36490)
0807 08:37:27.809621 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:27.809674 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 08:37:27.809929 29638 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.61:38385: Network error: Client connection negotiation failed: client connection to 127.9.174.61:38385: connect: Connection refused (error 111)
W20260807 08:37:27.826858 29647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:27.827307 29647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:27.827395 29647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:27.831584 29647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:27.831700 29647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:27.831737 29647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:27.831779 29647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:27.831816 29647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:27.837741 29647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:39749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:33937,127.9.174.61:38385,127.9.174.60:32949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.61.keytab
--principal=kudu/127.9.174.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.61:38385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.174.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:27.839288 29647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:27.840605 29647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:27.852056 29660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:27.852237 29647 server_base.cc:1034] running on GCE node
W20260807 08:37:27.852073 29662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:27.852459 29657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:27.853230 29647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:27.854857 29647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:27.856093 29647 hybrid_clock.cc:648] HybridClock initialized: now 1786091847856060 us; error 63 us; skew 500 ppm
Aug 07 08:37:27 dist-test-slave-h3ft krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:27.859862 29647 init.cc:377] Logged in from keytab as kudu/127.9.174.61@KRBTEST.COM (short username kudu)
I20260807 08:37:27.861193 29647 webserver.cc:460] Webserver started at http://127.9.174.61:35771/ using document root <none> and password file <none>
I20260807 08:37:27.861955 29647 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:27.862064 29647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:27.862357 29647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:27.865104 29647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/data/instance:
uuid: "64071a0def2a414f848bd285dbba4181"
format_stamp: "Formatted at 2026-08-07 08:37:27 on dist-test-slave-h3ft"
server_key: "3a2a1614cb70fbc74f31b0bea24b07a6"
server_key_iv: "61cb3e5482d6495f45dfa5b3a73f3afe"
server_key_version: "encryptionkey@0"
I20260807 08:37:27.865808 29647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/wal/instance:
uuid: "64071a0def2a414f848bd285dbba4181"
format_stamp: "Formatted at 2026-08-07 08:37:27 on dist-test-slave-h3ft"
server_key: "3a2a1614cb70fbc74f31b0bea24b07a6"
server_key_iv: "61cb3e5482d6495f45dfa5b3a73f3afe"
server_key_version: "encryptionkey@0"
I20260807 08:37:27.870854 29647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:37:27.874022 29669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:27.875754 29647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:27.875916 29647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/wal
uuid: "64071a0def2a414f848bd285dbba4181"
format_stamp: "Formatted at 2026-08-07 08:37:27 on dist-test-slave-h3ft"
server_key: "3a2a1614cb70fbc74f31b0bea24b07a6"
server_key_iv: "61cb3e5482d6495f45dfa5b3a73f3afe"
server_key_version: "encryptionkey@0"
I20260807 08:37:27.876067 29647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:27.901106 29638 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } attempt: 3
I20260807 08:37:27.901650 29647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:27.902362 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.901871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:38385 (local address 127.0.0.1:36498)
0807 08:37:27.902138 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:27.902232 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260807 08:37:27.903309 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.903084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:38385 (local address 127.0.0.1:36504)
0807 08:37:27.903192 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:27.903244 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 08:37:27.903478 29638 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.61:38385: Network error: Client connection negotiation failed: client connection to 127.9.174.61:38385: connect: Connection refused (error 111)
I20260807 08:37:27.905903 29647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:27.906143 29647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:27.915390 29647 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.61:38385
I20260807 08:37:27.915470 29744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.61:38385 every 8 connection(s)
I20260807 08:37:27.916712 29647 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/data/info.pb
I20260807 08:37:27.920660 29745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:27.921002 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 29647
I20260807 08:37:27.921164 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-1/wal/instance
I20260807 08:37:27.921520 9912 external_mini_cluster.cc:1428] Setting key 10003c3ee15ad1ed651b9a9488612d8c
I20260807 08:37:27.926702 29745 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.174.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.60@KRBTEST.COM" created.
Aug 07 08:37:27 dist-test-slave-h3ft krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.61@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:27.946785 29643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.929997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57866 (local address 127.9.174.62:33937)
0807 08:37:27.930261 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:27.930270 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:27.931460 (+ 1190us) server_negotiation.cc:407] Connection header received
0807 08:37:27.932724 (+ 1264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:27.932730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:27.932824 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:27.932958 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:27.934595 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.935315 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.937105 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.937599 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.937746 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:27.941233 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:27.941240 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:27.941245 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:27.941290 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:27.943519 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:27.944235 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:27.944243 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:27.944247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:27.944378 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:27.945036 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:27.945041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:27.945043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:27.945369 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:27.945504 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:27.946259 (+ 755us) server_negotiation.cc:299] Negotiation successful
0807 08:37:27.946472 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 08:37:27.947850 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.930533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:33937 (local address 127.0.0.1:57866)
0807 08:37:27.931230 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:27.931295 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:27.932428 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:27.933194 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:27.933202 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:27.933633 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:27.934368 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:27.934385 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.935534 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.935537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:27.936896 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:27.936909 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.937818 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.937822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:27.938035 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:27.938847 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:27.938867 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:27.940897 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:27.943693 (+ 2796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:27.943702 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:27.943721 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:27.944095 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:27.944521 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:27.944524 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:27.944525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:27.944907 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:27.945544 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:27.945551 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:27.945915 (+ 364us) client_negotiation.cc:769] Sending connection context
0807 08:37:27.946143 (+ 228us) client_negotiation.cc:240] Negotiation successful
0807 08:37:27.946530 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":217,"threads_started":1}
I20260807 08:37:27.949869 29745 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:37:27 dist-test-slave-h3ft krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.61@KRBTEST.COM for kudu/127.9.174.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.60.keytab.
Entry for principal kudu/127.9.174.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.60.keytab.
I20260807 08:37:27.959734 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.60:32949
--webserver_interface=127.9.174.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.60.keytab
--principal=kudu/127.9.174.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:39749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:33937,127.9.174.61:38385,127.9.174.60:32949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:37:27.962558 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.950567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:38385 (local address 127.0.0.1:36508)
0807 08:37:27.950727 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:27.950745 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:27.950841 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:27.951777 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:27.951779 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:27.951969 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:27.952125 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:27.952132 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.953642 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.953645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:27.954402 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:27.954412 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.955361 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.955364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:27.955517 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:27.956217 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:27.956232 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:27.957874 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:27.960479 (+ 2605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:27.960486 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:27.960491 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:27.960795 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:27.961161 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:27.961168 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:27.961172 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:27.961286 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:27.961896 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:27.961902 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:27.962037 (+ 135us) client_negotiation.cc:769] Sending connection context
0807 08:37:27.962165 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 08:37:27.962380 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":26}
I20260807 08:37:27.962809 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.950777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36508 (local address 127.9.174.61:38385)
0807 08:37:27.951312 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:27.951324 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:27.951349 (+ 25us) server_negotiation.cc:407] Connection header received
0807 08:37:27.951390 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:27.951395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:27.951591 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:27.951728 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:27.952349 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.953502 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.954535 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.955171 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.955350 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:27.958094 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:27.958109 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:27.958134 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:27.958168 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:27.960357 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:27.960944 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:27.960950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:27.960951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:27.961045 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:27.961415 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:27.961419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:27.961420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:27.961765 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:27.961895 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:27.962348 (+ 453us) server_negotiation.cc:299] Negotiation successful
0807 08:37:27.962554 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":408,"thread_start_us":294,"threads_started":1}
I20260807 08:37:27.965284 29745 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:37:27.966383 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.965933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:32949 (local address 127.0.0.1:33106)
0807 08:37:27.966148 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:27.966266 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:32949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260807 08:37:27.966789 29671 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.174.60:32949: connect: Connection refused (error 111)
I20260807 08:37:27.968075 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.967739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:32949 (local address 127.0.0.1:33110)
0807 08:37:27.967895 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:27.967978 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:32949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 08:37:27.968293 29745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:32949: Network error: Client connection negotiation failed: client connection to 127.9.174.60:32949: connect: Connection refused (error 111)
I20260807 08:37:28.017699 29745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } attempt: 1
I20260807 08:37:28.019097 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.018598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:32949 (local address 127.0.0.1:33114)
0807 08:37:28.018823 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:28.018928 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:32949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 08:37:28.020030 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.019757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:32949 (local address 127.0.0.1:33130)
0807 08:37:28.019893 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:28.019958 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:32949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 08:37:28.020174 29745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:32949: Network error: Client connection negotiation failed: client connection to 127.9.174.60:32949: connect: Connection refused (error 111)
I20260807 08:37:28.046696 29638 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } attempt: 4
Aug 07 08:37:28 dist-test-slave-h3ft krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for kudu/127.9.174.61@KRBTEST.COM
I20260807 08:37:28.059602 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.047737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36516 (local address 127.9.174.61:38385)
0807 08:37:28.047951 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:28.047959 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:28.048053 (+ 94us) server_negotiation.cc:407] Connection header received
0807 08:37:28.048205 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:28.048210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:28.048285 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:28.048394 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:28.049489 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.050904 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.051984 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.052409 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.052528 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.055587 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:28.055590 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:28.055593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:28.055625 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:28.057393 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.058020 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.058025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.058030 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.058151 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.058503 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.058506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.058507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.058756 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:28.058847 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:28.059277 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 08:37:28.059430 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 08:37:28.059715 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.047632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:38385 (local address 127.0.0.1:36516)
0807 08:37:28.047955 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:28.047971 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:28.048092 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:28.048472 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:28.048475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:28.048701 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:28.049318 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:28.049328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.051133 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.051137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:28.051788 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:28.051800 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.052569 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.052572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:28.052693 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.053405 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:28.053419 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:28.055394 (+ 1975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:28.057531 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:28.057535 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:28.057537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:28.057871 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:28.058256 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:28.058260 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:28.058264 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:28.058388 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:28.058864 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:28.058870 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:28.059066 (+ 196us) client_negotiation.cc:769] Sending connection context
0807 08:37:28.059209 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 08:37:28.059438 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202}
I20260807 08:37:28.061151 29638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:37:28.062227 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.061805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:32949 (local address 127.0.0.1:33138)
0807 08:37:28.062002 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:28.062101 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:32949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 08:37:28.063195 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.062883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:32949 (local address 127.0.0.1:33144)
0807 08:37:28.063072 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:28.063126 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:32949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 08:37:28.063361 29638 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:32949: Network error: Client connection negotiation failed: client connection to 127.9.174.60:32949: connect: Connection refused (error 111)
I20260807 08:37:28.088423 29745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } attempt: 2
I20260807 08:37:28.089808 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.089371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:32949 (local address 127.0.0.1:33150)
0807 08:37:28.089606 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:28.089687 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:32949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260807 08:37:28.090793 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.090517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:32949 (local address 127.0.0.1:33164)
0807 08:37:28.090652 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:28.090717 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:32949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 08:37:28.090956 29745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:32949: Network error: Client connection negotiation failed: client connection to 127.9.174.60:32949: connect: Connection refused (error 111)
W20260807 08:37:28.104799 29755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:28.105219 29755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:28.105320 29755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:28.109797 29755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:28.109916 29755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:28.109953 29755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:28.109995 29755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:28.110035 29755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:28.116480 29755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:39749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:33937,127.9.174.61:38385,127.9.174.60:32949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.60.keytab
--principal=kudu/127.9.174.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.60:32949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.174.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:28.118068 29755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:28.119499 29755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:37:28.122560 29638 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } attempt: 1
I20260807 08:37:28.123759 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.123341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:32949 (local address 127.0.0.1:33180)
0807 08:37:28.123548 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:28.123633 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:32949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 08:37:28.124701 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.124382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:32949 (local address 127.0.0.1:33196)
0807 08:37:28.124529 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:28.124599 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:32949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 08:37:28.124893 29638 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:32949: Network error: Client connection negotiation failed: client connection to 127.9.174.60:32949: connect: Connection refused (error 111)
W20260807 08:37:28.130347 29762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:28.130491 29763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:28.130359 29765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:28.130759 29755 server_base.cc:1034] running on GCE node
I20260807 08:37:28.131606 29755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:28.133067 29755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:28.134346 29755 hybrid_clock.cc:648] HybridClock initialized: now 1786091848134308 us; error 50 us; skew 500 ppm
Aug 07 08:37:28 dist-test-slave-h3ft krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091848, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:28.137939 29755 init.cc:377] Logged in from keytab as kudu/127.9.174.60@KRBTEST.COM (short username kudu)
I20260807 08:37:28.139180 29755 webserver.cc:460] Webserver started at http://127.9.174.60:41317/ using document root <none> and password file <none>
I20260807 08:37:28.139957 29755 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:28.140058 29755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:28.140345 29755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:28.142942 29755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/data/instance:
uuid: "a6ec78ad440a4b818f0cbfe3ab52171e"
format_stamp: "Formatted at 2026-08-07 08:37:28 on dist-test-slave-h3ft"
server_key: "f4a6bf5ac31f49b217ad8d90a8fb1f3f"
server_key_iv: "756e1d0f0e5e1af00cde0ecf914802d8"
server_key_version: "encryptionkey@0"
I20260807 08:37:28.143738 29755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/wal/instance:
uuid: "a6ec78ad440a4b818f0cbfe3ab52171e"
format_stamp: "Formatted at 2026-08-07 08:37:28 on dist-test-slave-h3ft"
server_key: "f4a6bf5ac31f49b217ad8d90a8fb1f3f"
server_key_iv: "756e1d0f0e5e1af00cde0ecf914802d8"
server_key_version: "encryptionkey@0"
I20260807 08:37:28.148370 29755 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 08:37:28.151404 29777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:28.152951 29755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:37:28.153131 29755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/wal
uuid: "a6ec78ad440a4b818f0cbfe3ab52171e"
format_stamp: "Formatted at 2026-08-07 08:37:28 on dist-test-slave-h3ft"
server_key: "f4a6bf5ac31f49b217ad8d90a8fb1f3f"
server_key_iv: "756e1d0f0e5e1af00cde0ecf914802d8"
server_key_version: "encryptionkey@0"
I20260807 08:37:28.153276 29755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:28.164127 29755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:28.168092 29755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:28.168354 29755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:28.177310 29755 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.60:32949
I20260807 08:37:28.177345 29860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.60:32949 every 8 connection(s)
I20260807 08:37:28.178634 29755 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/data/info.pb
I20260807 08:37:28.180589 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 29755
I20260807 08:37:28.180733 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091655732880-9912-0/minicluster-data/master-2/wal/instance
I20260807 08:37:28.181118 9912 external_mini_cluster.cc:1428] Setting key de8c9570e93563983d87a7ba82d13515
I20260807 08:37:28.182173 29745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } attempt: 3
I20260807 08:37:28.182575 29862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:28.188488 29862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:37:28 dist-test-slave-h3ft krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.61@KRBTEST.COM for kudu/127.9.174.60@KRBTEST.COM
Aug 07 08:37:28 dist-test-slave-h3ft krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.60@KRBTEST.COM
I20260807 08:37:28.192301 29638 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } attempt: 2
Aug 07 08:37:28 dist-test-slave-h3ft krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091848, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.60@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:28.198323 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.182704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:32949 (local address 127.0.0.1:33218)
0807 08:37:28.182899 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:28.182914 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:28.183040 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:28.185125 (+ 2085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:28.185129 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:28.185383 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:28.185539 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:28.185550 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.187316 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.187321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:28.188215 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:28.188230 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.189272 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.189276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:28.189380 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.190112 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:28.190134 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:28.192174 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:28.195756 (+ 3582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:28.195760 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:28.195762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:28.196070 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:28.196513 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:28.196516 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:28.196518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:28.196651 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:28.197645 (+ 994us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:28.197648 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:28.197786 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 08:37:28.197874 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 08:37:28.198062 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
Aug 07 08:37:28 dist-test-slave-h3ft krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for kudu/127.9.174.60@KRBTEST.COM
I20260807 08:37:28.200428 29866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.183250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33218 (local address 127.9.174.60:32949)
0807 08:37:28.183611 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:28.183621 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:28.183639 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:37:28.184294 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:28.184302 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:28.184748 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:28.185678 (+ 930us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:28.185820 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.187144 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.188359 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.189098 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.189735 (+ 637us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.192770 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:28.192778 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:28.192794 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:28.192833 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:28.195618 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.196193 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.196196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.196197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.196379 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.196789 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.196793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.196795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.197514 (+ 719us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:28.198697 (+ 1183us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:28.198776 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 08:37:28.199692 (+ 916us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":833,"server-negotiator.queue_time_us":286,"spinlock_wait_cycles":2432,"thread_start_us":176,"threads_started":1}
I20260807 08:37:28.200429 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.182851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33208 (local address 127.9.174.60:32949)
0807 08:37:28.183408 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:28.183439 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:28.183496 (+ 57us) server_negotiation.cc:407] Connection header received
0807 08:37:28.184271 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:28.184289 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:28.184745 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:28.185127 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:28.185980 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.187109 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.188395 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.189128 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.189957 (+ 829us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.193101 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:28.193106 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:28.193111 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:28.193144 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:28.195565 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.196105 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.196113 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.196124 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.196900 (+ 776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.197257 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.197261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.197262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.197533 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:28.197758 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:28.198678 (+ 920us) server_negotiation.cc:299] Negotiation successful
0807 08:37:28.199086 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":367,"spinlock_wait_cycles":16256,"thread_start_us":156,"threads_started":1}
I20260807 08:37:28.204280 29643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.191179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57882 (local address 127.9.174.62:33937)
0807 08:37:28.191339 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:28.191344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:28.191786 (+ 442us) server_negotiation.cc:407] Connection header received
0807 08:37:28.192043 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:28.192046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:28.192103 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:28.192205 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:28.193236 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.194079 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.195505 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.195850 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.195953 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.199436 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:28.199440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:28.199443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:28.199471 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:28.201486 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.202349 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.202353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.202356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.202474 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.202798 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.202801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.202802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.203218 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:28.203327 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:28.203928 (+ 601us) server_negotiation.cc:299] Negotiation successful
0807 08:37:28.204071 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 08:37:28.204810 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.191111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:33937 (local address 127.0.0.1:57882)
0807 08:37:28.191637 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:28.191692 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:28.191891 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:28.192263 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:28.192269 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:28.192872 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:28.193063 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:28.193073 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.194227 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.194232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:28.195343 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:28.195357 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.196152 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.196157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:28.196286 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.196865 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:28.196907 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:28.199278 (+ 2371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:28.201604 (+ 2326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:28.201612 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:28.201618 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:28.202217 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:28.202577 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:28.202583 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:28.202588 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:28.202695 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:28.203389 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:28.203400 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:28.203573 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 08:37:28.204433 (+ 860us) client_negotiation.cc:240] Negotiation successful
0807 08:37:28.204623 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"spinlock_wait_cycles":11136,"thread_start_us":121,"threads_started":1}
I20260807 08:37:28.206205 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.192814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:32949 (local address 127.0.0.1:33234)
0807 08:37:28.193135 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:28.193631 (+ 496us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:28.193728 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:28.194291 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:28.194295 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:28.194806 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:28.194945 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:28.194955 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.196100 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.196106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:28.197186 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:28.197198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.197828 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.197831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:28.197917 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.198410 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:28.198438 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:28.200223 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:28.204365 (+ 4142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:28.204368 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:28.204370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:28.204639 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:28.205199 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:28.205203 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:28.205204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:28.205303 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:28.205686 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:28.205688 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:28.205794 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 08:37:28.205908 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 08:37:28.206049 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222}
I20260807 08:37:28.206238 29871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.193304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33234 (local address 127.9.174.60:32949)
0807 08:37:28.193695 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:28.193705 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:28.193720 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:37:28.193827 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:28.193830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:28.193871 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:28.193972 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:28.195131 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.195948 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.197336 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.197698 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.199070 (+ 1372us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.200370 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:28.200374 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:28.200377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:28.200401 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:28.204248 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.204757 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.204761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.204762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.204852 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.205406 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.205409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.205410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.205585 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:28.205663 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:28.206010 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 08:37:28.206109 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":145,"threads_started":1}
I20260807 08:37:28.206409 29745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181: Bootstrap starting.
I20260807 08:37:28.207993 29862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:37:28.210121 29638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d: Bootstrap starting.
I20260807 08:37:28.211299 29745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:28.212508 29745 log.cc:824] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:28.213263 29638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:28.214329 29638 log.cc:824] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:37:28 dist-test-slave-h3ft krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
Aug 07 08:37:28 dist-test-slave-h3ft krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091848, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.60@KRBTEST.COM for kudu/127.9.174.61@KRBTEST.COM
I20260807 08:37:28.215864 29745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181: No bootstrap required, opened a new log
I20260807 08:37:28.216692 29638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d: No bootstrap required, opened a new log
I20260807 08:37:28.219937 29643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.207692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57894 (local address 127.9.174.62:33937)
0807 08:37:28.207812 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:28.207818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:28.207832 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:37:28.207868 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:28.207871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:28.207914 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:28.208010 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:28.209180 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.209887 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.211142 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.211460 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.211564 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.215728 (+ 4164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:28.215733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:28.215736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:28.215767 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:28.217585 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.218186 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.218190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.218192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.218289 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.218647 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.218651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.218653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.218942 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:28.219145 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:28.219585 (+ 440us) server_negotiation.cc:299] Negotiation successful
0807 08:37:28.219746 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:37:28.220345 29638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } }
I20260807 08:37:28.220765 29638 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:28.220950 29745 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } }
I20260807 08:37:28.221205 29745 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:28.221299 29638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66605a02d4fb4380a9b2168b5e4c9b3d, State: Initialized, Role: FOLLOWER
I20260807 08:37:28.221335 29745 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64071a0def2a414f848bd285dbba4181, State: Initialized, Role: FOLLOWER
I20260807 08:37:28.221999 29638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } }
I20260807 08:37:28.222312 29745 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } }
I20260807 08:37:28.222882 29878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } } }
I20260807 08:37:28.223304 29878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:37:28.223925 29745 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:28.223914 29638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:28.224663 29877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } } }
I20260807 08:37:28.224864 29877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:37:28.226102 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.208646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:38385 (local address 127.0.0.1:36526)
0807 08:37:28.208883 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:28.208899 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:28.208980 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:28.209594 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:28.209597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:28.209778 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:28.209906 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:28.209917 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.211064 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.211069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:28.211885 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:28.211897 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.212748 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.212751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:28.212892 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.213373 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:28.213387 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:28.220733 (+ 7346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:28.223058 (+ 2325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:28.223061 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:28.223064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:28.223349 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:28.223721 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:28.223723 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:28.223725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:28.223822 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:28.225359 (+ 1537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:28.225362 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:28.225512 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 08:37:28.225616 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 08:37:28.225814 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260807 08:37:28.226701 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.209201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36526 (local address 127.9.174.61:38385)
0807 08:37:28.209339 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:28.209344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:28.209361 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:37:28.209405 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:28.209408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:28.209455 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:28.209549 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:28.210103 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.210885 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.212050 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.212442 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.212546 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.220944 (+ 8398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:28.220948 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:28.220951 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:28.220987 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:28.222858 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.223490 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.223494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.223496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.223588 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.223960 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.223963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.223965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.224261 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:28.224379 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:28.226383 (+ 2004us) server_negotiation.cc:299] Negotiation successful
0807 08:37:28.226551 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:37:28.229298 29862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:37:28 dist-test-slave-h3ft krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.61@KRBTEST.COM
W20260807 08:37:28.231397 29907 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:37:28.231492 29907 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:37:28.235327 29876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.222032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36532 (local address 127.9.174.61:38385)
0807 08:37:28.224115 (+ 2083us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:28.224125 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:28.224140 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:37:28.224177 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:28.224179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:28.224226 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:28.224324 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:28.225347 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.226247 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.227456 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.227801 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.227891 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.231230 (+ 3339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:28.231237 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:28.231241 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:28.231273 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:28.233095 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.233625 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.233629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.233631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.233737 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.234059 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.234062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.234064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.234326 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:28.234521 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:28.234966 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 08:37:28.235132 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1999,"thread_start_us":168,"threads_started":1}
Aug 07 08:37:28 dist-test-slave-h3ft krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091848, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.60@KRBTEST.COM for kudu/127.9.174.60@KRBTEST.COM
I20260807 08:37:28.236189 29647 master_runner.cc:186] Error getting master registration for 127.9.174.60:32949: OK, instance_id { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" instance_seqno: 1786091848174622 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:37:28.241091 29647 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 08:37:28.242116 29908 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:37:28.242259 29908 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:37:28.242378 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:37:28.241557 29522 master_runner.cc:186] Error getting master registration for 127.9.174.60:32949: OK, instance_id { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" instance_seqno: 1786091848174622 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:37:28.245529 29522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:37:28.249176 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.229866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:32949 (local address 127.0.0.1:33240)
0807 08:37:28.232661 (+ 2795us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:28.232677 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:28.232771 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:28.233111 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:28.233113 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:28.233316 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:28.233440 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:28.233448 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.234417 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.234420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:28.235195 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:28.235205 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.235895 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.235899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:28.235977 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.239210 (+ 3233us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:28.239226 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:28.241074 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:28.247313 (+ 6239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:28.247316 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:28.247319 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:28.247611 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:28.247930 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:28.247933 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:28.247934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:28.248024 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:28.248561 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:28.248566 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:28.248679 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 08:37:28.248775 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 08:37:28.248957 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2690}
I20260807 08:37:28.249774 29871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.231157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33240 (local address 127.9.174.60:32949)
0807 08:37:28.231278 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:28.231283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:28.232849 (+ 1566us) server_negotiation.cc:407] Connection header received
0807 08:37:28.232877 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:28.232879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:28.232923 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:28.232984 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:28.233578 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.234308 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.235315 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.235657 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.235763 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.241230 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:28.241233 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:28.241237 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:28.241263 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:28.247160 (+ 5897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.247729 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.247735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.247740 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.247834 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.248115 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.248118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.248120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.248411 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:28.248521 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:28.249438 (+ 917us) server_negotiation.cc:299] Negotiation successful
0807 08:37:28.249603 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:37:28.254123 29862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e: Bootstrap starting.
I20260807 08:37:28.259562 29862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:28.261349 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.246595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33242 (local address 127.9.174.60:32949)
0807 08:37:28.246770 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:28.246776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:28.250559 (+ 3783us) server_negotiation.cc:407] Connection header received
0807 08:37:28.250760 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:28.250763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:28.250816 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:28.250870 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:28.251968 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.252834 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.253993 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.254335 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.254417 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.256507 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:28.256513 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:28.256516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:28.256543 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:28.258702 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.259221 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.259225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.259227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.259323 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.259696 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.259700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.259701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.259958 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:28.260110 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:28.261014 (+ 904us) server_negotiation.cc:299] Negotiation successful
0807 08:37:28.261157 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 08:37:28.263499 29862 log.cc:824] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:28.270619 29862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e: No bootstrap required, opened a new log
I20260807 08:37:28.272534 29876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.248502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36542 (local address 127.9.174.61:38385)
0807 08:37:28.248647 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:28.248652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:28.260823 (+ 12171us) server_negotiation.cc:407] Connection header received
0807 08:37:28.260948 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:28.260951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:28.261056 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:28.261144 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:28.262446 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.263306 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.264594 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.265012 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.265089 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.268052 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:28.268056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:28.268060 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:28.268098 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:28.269889 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.270410 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.270415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.270417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.270515 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.271226 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.271230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.271231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.271479 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:28.271594 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:28.272199 (+ 605us) server_negotiation.cc:299] Negotiation successful
0807 08:37:28.272344 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 08:37:28.276752 29862 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } }
I20260807 08:37:28.277137 29862 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:28.277240 29862 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ec78ad440a4b818f0cbfe3ab52171e, State: Initialized, Role: FOLLOWER
I20260807 08:37:28.277908 29862 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } }
I20260807 08:37:28.279289 29643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:28.247956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57910 (local address 127.9.174.62:33937)
0807 08:37:28.248134 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:28.248138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:28.257087 (+ 8949us) server_negotiation.cc:407] Connection header received
0807 08:37:28.257736 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:28.257740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:28.257810 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:28.257873 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:28.262379 (+ 4506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.263284 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.264544 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:28.264917 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:28.265000 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:28.274854 (+ 9854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:28.274859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:28.274862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:28.274902 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:28.276760 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.277546 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.277551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.277553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.277655 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:28.278049 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:28.278053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:28.278055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:28.278335 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:28.278483 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:28.278901 (+ 418us) server_negotiation.cc:299] Negotiation successful
0807 08:37:28.279104 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 08:37:28.279907 29862 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:28.282771 29922 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } } }
I20260807 08:37:28.283219 29922 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:37:28.288623 29755 master_runner.cc:263] No leader master found from master a6ec78ad440a4b818f0cbfe3ab52171e
I20260807 08:37:28.288718 29755 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 08:37:28.293359 29938 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:37:28.293576 29938 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:37:28.346948 29878 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:28.347242 29878 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } }
I20260807 08:37:28.348630 29878 leader_election.cc:288] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66605a02d4fb4380a9b2168b5e4c9b3d (127.9.174.62:33937), a6ec78ad440a4b818f0cbfe3ab52171e (127.9.174.60:32949)
I20260807 08:37:28.349784 29821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64071a0def2a414f848bd285dbba4181" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" is_pre_election: true
I20260807 08:37:28.350198 29821 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64071a0def2a414f848bd285dbba4181 in term 0.
I20260807 08:37:28.350883 29671 leader_election.cc:302] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64071a0def2a414f848bd285dbba4181, a6ec78ad440a4b818f0cbfe3ab52171e; no voters:
I20260807 08:37:28.351591 29878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:28.351702 29878 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:28.351755 29878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:28.351764 29588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64071a0def2a414f848bd285dbba4181" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" is_pre_election: true
I20260807 08:37:28.352125 29588 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64071a0def2a414f848bd285dbba4181 in term 0.
I20260807 08:37:28.353581 29878 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } }
I20260807 08:37:28.354329 29588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64071a0def2a414f848bd285dbba4181" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d"
I20260807 08:37:28.354487 29821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64071a0def2a414f848bd285dbba4181" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e"
I20260807 08:37:28.354517 29588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:28.354640 29821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:28.356141 29821 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64071a0def2a414f848bd285dbba4181 in term 1.
I20260807 08:37:28.356141 29588 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64071a0def2a414f848bd285dbba4181 in term 1.
I20260807 08:37:28.356525 29673 leader_election.cc:302] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64071a0def2a414f848bd285dbba4181, 66605a02d4fb4380a9b2168b5e4c9b3d; no voters:
I20260807 08:37:28.356710 29878 leader_election.cc:288] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [CANDIDATE]: Term 1 election: Requested vote from peers 66605a02d4fb4380a9b2168b5e4c9b3d (127.9.174.62:33937), a6ec78ad440a4b818f0cbfe3ab52171e (127.9.174.60:32949)
I20260807 08:37:28.356909 29878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:28.357085 29878 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [term 1 LEADER]: Becoming Leader. State: Replica: 64071a0def2a414f848bd285dbba4181, State: Running, Role: LEADER
I20260807 08:37:28.357643 29878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } }
I20260807 08:37:28.363590 29878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64071a0def2a414f848bd285dbba4181. Latest consensus state: current_term: 1 leader_uuid: "64071a0def2a414f848bd285dbba4181" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } } }
I20260807 08:37:28.363847 29878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:28.364378 29947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:28.365571 29947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:28.377187 29821 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e [term 1 FOLLOWER]: Refusing update from remote peer 64071a0def2a414f848bd285dbba4181: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:28.378950 29941 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:28.385773 29922 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64071a0def2a414f848bd285dbba4181. Latest consensus state: current_term: 1 leader_uuid: "64071a0def2a414f848bd285dbba4181" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } } }
I20260807 08:37:28.385953 29922 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:37:28.387579 29588 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d [term 1 FOLLOWER]: Refusing update from remote peer 64071a0def2a414f848bd285dbba4181: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:28.388487 29941 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:28.392530 29941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "64071a0def2a414f848bd285dbba4181" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } } }
I20260807 08:37:28.392729 29941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:28.394011 29877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64071a0def2a414f848bd285dbba4181. Latest consensus state: current_term: 1 leader_uuid: "64071a0def2a414f848bd285dbba4181" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } } }
I20260807 08:37:28.394182 29877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:37:28.394344 29922 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "64071a0def2a414f848bd285dbba4181" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } } }
I20260807 08:37:28.394526 29922 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:37:28.394691 29952 mvcc.cc:205] Tried to move back new op lower bound from 7315832210935255040 to 7315832210885521408. Current Snapshot: MvccSnapshot[applied={T|T < 7315832210935255040}]
W20260807 08:37:28.395582 29551 tablet.cc:2369] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:37:28.397924 29953 mvcc.cc:205] Tried to move back new op lower bound from 7315832210935255040 to 7315832210885521408. Current Snapshot: MvccSnapshot[applied={T|T < 7315832210935255040}]
I20260807 08:37:28.402061 29947 catalog_manager.cc:1347] Generated new cluster ID: 2071d9a53ce040df8060a91bb8466efd
I20260807 08:37:28.402134 29947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:28.428587 29878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "64071a0def2a414f848bd285dbba4181" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } } }
I20260807 08:37:28.428792 29878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:28.430518 29877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "64071a0def2a414f848bd285dbba4181" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66605a02d4fb4380a9b2168b5e4c9b3d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33937 } } peers { permanent_uuid: "64071a0def2a414f848bd285dbba4181" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 38385 } } peers { permanent_uuid: "a6ec78ad440a4b818f0cbfe3ab52171e" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 32949 } } }
I20260807 08:37:28.430670 29877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:37:28.434176 29957 mvcc.cc:205] Tried to move back new op lower bound from 7315832210935255040 to 7315832210885521408. Current Snapshot: MvccSnapshot[applied={T|T < 7315832210935255040}]
I20260807 08:37:28.460041 29947 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:28.461766 29947 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:28.474908 29947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64071a0def2a414f848bd285dbba4181: Generated new TSK 0
I20260807 08:37:28.476184 29947 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:37:28.924624 29734 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.9.174.61:38385
I20260807 08:37:29.233470 29907 catalog_manager.cc:1259] Loaded cluster ID: 2071d9a53ce040df8060a91bb8466efd
I20260807 08:37:29.233606 29907 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d: loading cluster ID for follower catalog manager: success
I20260807 08:37:29.235685 29907 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d: acquiring CA information for follower catalog manager: success
I20260807 08:37:29.237309 29907 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 66605a02d4fb4380a9b2168b5e4c9b3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:37:29.295506 29938 catalog_manager.cc:1259] Loaded cluster ID: 2071d9a53ce040df8060a91bb8466efd
I20260807 08:37:29.295625 29938 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e: loading cluster ID for follower catalog manager: success
I20260807 08:37:29.297564 29938 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e: acquiring CA information for follower catalog manager: success
I20260807 08:37:29.298902 29938 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a6ec78ad440a4b818f0cbfe3ab52171e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:37:29.553737 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 29522
I20260807 08:37:29.570979 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 29647
I20260807 08:37:29.586908 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 29755
2026-08-07T08:37:29Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3604 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:29 dist-test-slave-h3ft krb5kdc[29978](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:29 dist-test-slave-h3ft krb5kdc[29978](info): set up 2 sockets
Aug 07 08:37:29 dist-test-slave-h3ft krb5kdc[29978](info): commencing operation
krb5kdc: starting...
W20260807 08:37:30.895993 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:37:30 dist-test-slave-h3ft krb5kdc[29978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:37:31.042965 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:42747
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:36541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:42747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:31.182085 29994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:31.182437 29994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:31.182513 29994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:31.186589 29994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:31.186685 29994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:31.186723 29994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:31.186780 29994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:31.186816 29994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:31.192338 29994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:36541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:42747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:42747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:31.193706 29994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:31.194989 29994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:37:31.204857 29994 server_base.cc:1034] running on GCE node
W20260807 08:37:31.204829 30007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:31.204815 30001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:31.204815 30002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:31.205822 29994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:31.207124 29994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:31.208312 29994 hybrid_clock.cc:648] HybridClock initialized: now 1786091851208288 us; error 40 us; skew 500 ppm
Aug 07 08:37:31 dist-test-slave-h3ft krb5kdc[29978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091851, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:31.211570 29994 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:37:31.212638 29994 webserver.cc:460] Webserver started at http://127.9.174.62:34957/ using document root <none> and password file <none>
I20260807 08:37:31.213313 29994 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:31.213409 29994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:31.213678 29994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:31.216171 29994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "4831ea2dcb2a45a2955e21b5c61f5718"
format_stamp: "Formatted at 2026-08-07 08:37:31 on dist-test-slave-h3ft"
server_key: "fd9ff696136c84d9985b67c2f36fe594"
server_key_iv: "00cd2b132c2223a76c27c19a388543c6"
server_key_version: "encryptionkey@0"
I20260807 08:37:31.216832 29994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "4831ea2dcb2a45a2955e21b5c61f5718"
format_stamp: "Formatted at 2026-08-07 08:37:31 on dist-test-slave-h3ft"
server_key: "fd9ff696136c84d9985b67c2f36fe594"
server_key_iv: "00cd2b132c2223a76c27c19a388543c6"
server_key_version: "encryptionkey@0"
I20260807 08:37:31.221201 29994 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 08:37:31.224116 30016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:31.225479 29994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:31.225625 29994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "4831ea2dcb2a45a2955e21b5c61f5718"
format_stamp: "Formatted at 2026-08-07 08:37:31 on dist-test-slave-h3ft"
server_key: "fd9ff696136c84d9985b67c2f36fe594"
server_key_iv: "00cd2b132c2223a76c27c19a388543c6"
server_key_version: "encryptionkey@0"
I20260807 08:37:31.225744 29994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:31.251924 29994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:31.256278 29994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:31.256544 29994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:31.266570 29994 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:42747
I20260807 08:37:31.266610 30094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:42747 every 8 connection(s)
I20260807 08:37:31.267953 29994 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:31.271684 30095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:31.272336 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 29994
I20260807 08:37:31.272483 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:37:31.272835 9912 external_mini_cluster.cc:1428] Setting key d7b5dcbc3946aef3b2714de8d945cfbe
I20260807 08:37:31.280769 30095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718: Bootstrap starting.
I20260807 08:37:31.284315 30095 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718: Neither blocks nor log segments found. Creating new log.
Aug 07 08:37:31 dist-test-slave-h3ft krb5kdc[29978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:31.285441 30095 log.cc:824] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:31.288147 30095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718: No bootstrap required, opened a new log
I20260807 08:37:31.290845 30098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:31.274583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49068 (local address 127.9.174.62:42747)
0807 08:37:31.275080 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:31.275106 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:31.275159 (+ 53us) server_negotiation.cc:407] Connection header received
0807 08:37:31.276097 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:31.276125 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:31.276625 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:31.277049 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:31.277890 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.279571 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.280795 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.281631 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.281852 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:31.285550 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:31.285556 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:31.285589 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:31.285626 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:31.287572 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:31.288083 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:31.288088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:31.288093 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:31.288211 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:31.288669 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:31.288672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:31.288673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:31.289119 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:31.289290 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:31.289632 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 08:37:31.289985 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":115,"threads_started":1}
I20260807 08:37:31.292764 30095 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4831ea2dcb2a45a2955e21b5c61f5718" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42747 } }
I20260807 08:37:31.293085 30095 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:31.293147 30095 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4831ea2dcb2a45a2955e21b5c61f5718, State: Initialized, Role: FOLLOWER
I20260807 08:37:31.293634 30095 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4831ea2dcb2a45a2955e21b5c61f5718" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42747 } }
I20260807 08:37:31.293797 30095 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:31.293926 30095 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:31.294051 30095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:31.294595 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:37:31.295279 30095 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4831ea2dcb2a45a2955e21b5c61f5718" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42747 } }
I20260807 08:37:31.295648 30095 leader_election.cc:302] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4831ea2dcb2a45a2955e21b5c61f5718; no voters:
I20260807 08:37:31.296041 30095 leader_election.cc:288] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:31.296149 30101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:31.296391 30101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [term 1 LEADER]: Becoming Leader. State: Replica: 4831ea2dcb2a45a2955e21b5c61f5718, State: Running, Role: LEADER
I20260807 08:37:31.296800 30101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4831ea2dcb2a45a2955e21b5c61f5718" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42747 } }
I20260807 08:37:31.297462 30095 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:31.298831 30103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4831ea2dcb2a45a2955e21b5c61f5718" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4831ea2dcb2a45a2955e21b5c61f5718" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42747 } } }
I20260807 08:37:31.298869 30104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4831ea2dcb2a45a2955e21b5c61f5718. Latest consensus state: current_term: 1 leader_uuid: "4831ea2dcb2a45a2955e21b5c61f5718" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4831ea2dcb2a45a2955e21b5c61f5718" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42747 } } }
I20260807 08:37:31.299031 30103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:31.299041 30104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:31.299485 30113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:31.303260 30113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:31.309015 30113 catalog_manager.cc:1347] Generated new cluster ID: d59f72349deb475cb14af1a7453c44ae
I20260807 08:37:31.309088 30113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:31.320421 30113 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:31.321554 30113 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:31.328277 30113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4831ea2dcb2a45a2955e21b5c61f5718: Generated new TSK 0
I20260807 08:37:31.329084 30113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:37:31.352316 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 29994
2026-08-07T08:37:31Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1762 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:31 dist-test-slave-h3ft krb5kdc[30130](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:31 dist-test-slave-h3ft krb5kdc[30130](info): set up 2 sockets
Aug 07 08:37:31 dist-test-slave-h3ft krb5kdc[30130](info): commencing operation
krb5kdc: starting...
W20260807 08:37:32.621179 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:37:32 dist-test-slave-h3ft krb5kdc[30130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:37:32.772146 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:45261
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:45261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:32.909868 30147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:32.910399 30147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:32.910487 30147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:32.914855 30147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:32.914981 30147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:32.915045 30147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:32.915089 30147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:32.915122 30147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:32.921285 30147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:45261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:45261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:32.922788 30147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:32.924170 30147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:32.934985 30159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:32.934993 30161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:32.935109 30147 server_base.cc:1034] running on GCE node
W20260807 08:37:32.934985 30157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:32.936247 30147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:32.937780 30147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:32.939041 30147 hybrid_clock.cc:648] HybridClock initialized: now 1786091852938982 us; error 39 us; skew 500 ppm
Aug 07 08:37:32 dist-test-slave-h3ft krb5kdc[30130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091852, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:32.942670 30147 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:37:32.943966 30147 webserver.cc:460] Webserver started at http://127.9.174.62:38961/ using document root <none> and password file <none>
I20260807 08:37:32.944710 30147 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:32.944813 30147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:32.945099 30147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:32.947820 30147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "011c5d96f7874fe5b6bf66fe2a7adc11"
format_stamp: "Formatted at 2026-08-07 08:37:32 on dist-test-slave-h3ft"
server_key: "81b24a94c54ce3c4ace2b179c39d4f1e"
server_key_iv: "de7bd4b8e2a46e462af8728c63a7f38f"
server_key_version: "encryptionkey@0"
I20260807 08:37:32.948583 30147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "011c5d96f7874fe5b6bf66fe2a7adc11"
format_stamp: "Formatted at 2026-08-07 08:37:32 on dist-test-slave-h3ft"
server_key: "81b24a94c54ce3c4ace2b179c39d4f1e"
server_key_iv: "de7bd4b8e2a46e462af8728c63a7f38f"
server_key_version: "encryptionkey@0"
I20260807 08:37:32.953578 30147 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:37:32.956903 30174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:32.958640 30147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:37:32.958838 30147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "011c5d96f7874fe5b6bf66fe2a7adc11"
format_stamp: "Formatted at 2026-08-07 08:37:32 on dist-test-slave-h3ft"
server_key: "81b24a94c54ce3c4ace2b179c39d4f1e"
server_key_iv: "de7bd4b8e2a46e462af8728c63a7f38f"
server_key_version: "encryptionkey@0"
I20260807 08:37:32.959055 30147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:32.996593 30147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:33.001539 30147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:33.001832 30147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:33.010766 30147 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:45261
I20260807 08:37:33.011147 30259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:45261 every 8 connection(s)
I20260807 08:37:33.012138 30147 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:33.015626 30261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:33.022269 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 30147
I20260807 08:37:33.022457 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:37:33.022607 30261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: Bootstrap starting.
I20260807 08:37:33.022857 9912 external_mini_cluster.cc:1428] Setting key ab9860beef66c9ee86c89b53e9b76534
I20260807 08:37:33.025754 30261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:33.026640 30261 log.cc:824] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:33.028815 30261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: No bootstrap required, opened a new log
Aug 07 08:37:33 dist-test-slave-h3ft krb5kdc[30130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:33.033232 30261 raft_consensus.cc:348] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } }
I20260807 08:37:33.033604 30261 raft_consensus.cc:374] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:33.033705 30261 raft_consensus.cc:729] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011c5d96f7874fe5b6bf66fe2a7adc11, State: Initialized, Role: FOLLOWER
I20260807 08:37:33.034240 30261 consensus_queue.cc:260] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } }
I20260807 08:37:33.034433 30261 raft_consensus.cc:388] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:33.034551 30261 raft_consensus.cc:482] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:33.034687 30261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:33.035874 30261 raft_consensus.cc:504] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } }
I20260807 08:37:33.036341 30261 leader_election.cc:302] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 011c5d96f7874fe5b6bf66fe2a7adc11; no voters:
I20260807 08:37:33.036768 30261 leader_election.cc:288] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:33.036962 30267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:33.037240 30267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 1 LEADER]: Becoming Leader. State: Replica: 011c5d96f7874fe5b6bf66fe2a7adc11, State: Running, Role: LEADER
I20260807 08:37:33.037703 30267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } }
I20260807 08:37:33.038414 30261 sys_catalog.cc:564] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:33.039840 30269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 011c5d96f7874fe5b6bf66fe2a7adc11. Latest consensus state: current_term: 1 leader_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } } }
I20260807 08:37:33.039844 30268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } } }
I20260807 08:37:33.040017 30269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:33.040127 30268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:33.040210 30265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.024671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41830 (local address 127.9.174.62:45261)
0807 08:37:33.025197 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:33.025225 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:33.025267 (+ 42us) server_negotiation.cc:407] Connection header received
0807 08:37:33.026102 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:33.026124 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:33.026524 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:33.026900 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:33.027814 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.028899 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.029789 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.030285 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.030488 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.033682 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:33.033689 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:33.033716 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:33.033755 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:33.036043 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.036627 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.036634 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.036640 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.036816 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.037217 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.037221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.037223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.037813 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:33.038022 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:33.038551 (+ 529us) server_negotiation.cc:299] Negotiation successful
0807 08:37:33.038931 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":193,"threads_started":1}
W20260807 08:37:33.044415 30287 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:37:33.044514 30287 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:37:33.044579 30282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:33.045610 30282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:33.052284 30282 catalog_manager.cc:1347] Generated new cluster ID: 7d4fb95049ee411183dfb9b75bea2c88
I20260807 08:37:33.052377 30282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:33.062119 30282 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:33.063521 30282 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:33.069496 30282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: Generated new TSK 0
I20260807 08:37:33.070361 30282 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:37:33.111836 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:45261
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:33.260308 30297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:33.260643 30297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:33.260690 30297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:33.265126 30297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:33.265197 30297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:33.265292 30297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:37:33.270915 30297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:45261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:33.272315 30297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:33.273653 30297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:33.285426 30307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:33.285431 30305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:33.285422 30304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:33.285751 30297 server_base.cc:1034] running on GCE node
I20260807 08:37:33.286346 30297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:33.287130 30297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:33.288295 30297 hybrid_clock.cc:648] HybridClock initialized: now 1786091853288263 us; error 46 us; skew 500 ppm
Aug 07 08:37:33 dist-test-slave-h3ft krb5kdc[30130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:33.292727 30297 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:37:33.293795 30297 webserver.cc:460] Webserver started at http://127.9.174.1:44597/ using document root <none> and password file <none>
I20260807 08:37:33.294510 30297 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:33.294610 30297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:33.294862 30297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:33.297407 30297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "94499e8e1b714946bf6d45bcfeafdb86"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h3ft"
server_key: "288fa9ba02a1ab11d040bca02c663fb2"
server_key_iv: "a8299ab9e4afec4df99a0c0aab22195c"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.298136 30297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "94499e8e1b714946bf6d45bcfeafdb86"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h3ft"
server_key: "288fa9ba02a1ab11d040bca02c663fb2"
server_key_iv: "a8299ab9e4afec4df99a0c0aab22195c"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.302959 30297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 08:37:33.305930 30315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.307655 30297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:33.307919 30297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "94499e8e1b714946bf6d45bcfeafdb86"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h3ft"
server_key: "288fa9ba02a1ab11d040bca02c663fb2"
server_key_iv: "a8299ab9e4afec4df99a0c0aab22195c"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.308053 30297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:33.323297 30297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:33.327273 30297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:33.327518 30297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:33.328315 30297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:33.329355 30297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:33.329429 30297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.329504 30297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:33.329555 30297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.340503 30297 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:42079
I20260807 08:37:33.340549 30486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:42079 every 8 connection(s)
I20260807 08:37:33.341600 30297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:33.342664 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 30297
I20260807 08:37:33.342832 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:37:33.343309 9912 external_mini_cluster.cc:1428] Setting key 02a58390288b813bfa6a968a064c1598
Aug 07 08:37:33 dist-test-slave-h3ft krb5kdc[30130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
I20260807 08:37:33.358018 30265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.343894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:59381 (local address 127.9.174.62:45261)
0807 08:37:33.344089 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:33.344093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:33.345299 (+ 1206us) server_negotiation.cc:407] Connection header received
0807 08:37:33.346830 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:33.346834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:33.346912 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:33.347040 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:33.348432 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.349046 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.350296 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.350558 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.350678 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.353816 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:33.353820 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:33.353823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:33.353853 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:33.355477 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.356024 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.356028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.356030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.356122 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.356560 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.356568 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.356572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.356824 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:33.356952 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:33.357551 (+ 599us) server_negotiation.cc:299] Negotiation successful
0807 08:37:33.357714 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
I20260807 08:37:33.359253 30489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.344473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:59381)
0807 08:37:33.345140 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:33.345190 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:33.346599 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:33.347253 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:33.347261 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:33.347691 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:33.348266 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.348282 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.349182 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.349185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.350161 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.350171 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.350735 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.350740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.351050 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.351853 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:33.351871 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:33.353616 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:33.355604 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:33.355610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:33.355630 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:33.355915 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:33.356219 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:33.356222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:33.356224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:33.356398 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:33.356938 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:33.356946 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:33.357264 (+ 318us) client_negotiation.cc:769] Sending connection context
0807 08:37:33.357505 (+ 241us) client_negotiation.cc:240] Negotiation successful
0807 08:37:33.357912 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":176,"threads_started":1}
I20260807 08:37:33.360901 30487 heartbeater.cc:347] Connected to a master server at 127.9.174.62:45261
I20260807 08:37:33.361239 30487 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:33.361884 30487 heartbeater.cc:511] Master 127.9.174.62:45261 requested a full tablet report, sending...
I20260807 08:37:33.363925 30202 ts_manager.cc:194] Registered new tserver with Master: 94499e8e1b714946bf6d45bcfeafdb86 (127.9.174.1:42079)
I20260807 08:37:33.365350 30202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:59381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:37:33.378732 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:45261
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:33.512441 30495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:33.512804 30495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:33.512908 30495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:33.517205 30495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:33.517295 30495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:33.517386 30495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:37:33.523079 30495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:45261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:33.524545 30495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:33.525870 30495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:33.538025 30501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:33.538009 30504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:33.538018 30502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:33.538337 30495 server_base.cc:1034] running on GCE node
I20260807 08:37:33.538885 30495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:33.539983 30495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:33.541170 30495 hybrid_clock.cc:648] HybridClock initialized: now 1786091853541150 us; error 48 us; skew 500 ppm
Aug 07 08:37:33 dist-test-slave-h3ft krb5kdc[30130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:33.544864 30495 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:37:33.546088 30495 webserver.cc:460] Webserver started at http://127.9.174.2:44541/ using document root <none> and password file <none>
I20260807 08:37:33.546836 30495 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:33.546937 30495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:33.547317 30495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:33.550001 30495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "83d9331d2ca54303a1f19c963d20017a"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h3ft"
server_key: "36c65ec9cc60e4d23b538bb9ac9ea972"
server_key_iv: "56f38d67babe411cb206a94cbdf76fd9"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.550720 30495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "83d9331d2ca54303a1f19c963d20017a"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h3ft"
server_key: "36c65ec9cc60e4d23b538bb9ac9ea972"
server_key_iv: "56f38d67babe411cb206a94cbdf76fd9"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.555745 30495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 08:37:33.558774 30513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.560326 30495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:33.560491 30495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "83d9331d2ca54303a1f19c963d20017a"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h3ft"
server_key: "36c65ec9cc60e4d23b538bb9ac9ea972"
server_key_iv: "56f38d67babe411cb206a94cbdf76fd9"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.560628 30495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:33.581398 30495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:33.585232 30495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:33.585471 30495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:33.586251 30495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:33.587332 30495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:33.587386 30495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.587456 30495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:33.587495 30495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.598631 30495 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:41893
I20260807 08:37:33.598708 30678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:41893 every 8 connection(s)
I20260807 08:37:33.599884 30495 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:33.608471 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 30495
I20260807 08:37:33.608623 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:37:33.608984 9912 external_mini_cluster.cc:1428] Setting key 1cec74e3e64acef81179a19386b48358
Aug 07 08:37:33 dist-test-slave-h3ft krb5kdc[30130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:33.617391 30265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.602299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:45919 (local address 127.9.174.62:45261)
0807 08:37:33.602509 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:33.602513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:33.603519 (+ 1006us) server_negotiation.cc:407] Connection header received
0807 08:37:33.605152 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:33.605156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:33.605236 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:33.605347 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:33.606833 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.607511 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.609165 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.609417 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.609570 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.613009 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:33.613013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:33.613015 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:33.613044 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:33.614970 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.615558 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.615562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.615564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.615678 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.616088 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.616093 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.616095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.616320 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:33.616432 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:33.617034 (+ 602us) server_negotiation.cc:299] Negotiation successful
0807 08:37:33.617193 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 08:37:33.618429 30681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.602737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:45919)
0807 08:37:33.603316 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:33.603368 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:33.604881 (+ 1513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:33.605571 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:33.605578 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:33.606046 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:33.606653 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.606669 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.607656 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.607660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.609013 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.609030 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.609645 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.609653 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.609872 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.610730 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:33.610756 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:33.612787 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:33.615122 (+ 2335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:33.615128 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:33.615144 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:33.615454 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:33.615799 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:33.615802 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:33.615804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:33.615982 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:33.616434 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:33.616444 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:33.616763 (+ 319us) client_negotiation.cc:769] Sending connection context
0807 08:37:33.616981 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 08:37:33.617382 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":192,"threads_started":1}
I20260807 08:37:33.620052 30679 heartbeater.cc:347] Connected to a master server at 127.9.174.62:45261
I20260807 08:37:33.620368 30679 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:33.620992 30679 heartbeater.cc:511] Master 127.9.174.62:45261 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
I20260807 08:37:33.622471 30202 ts_manager.cc:194] Registered new tserver with Master: 83d9331d2ca54303a1f19c963d20017a (127.9.174.2:41893)
I20260807 08:37:33.623247 30202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:45919
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:37:33.644517 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:45261
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:33.781994 30688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:33.782354 30688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:33.782403 30688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:33.786700 30688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:33.786793 30688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:33.786861 30688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:37:33.792949 30688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:45261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:33.794327 30688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:33.795621 30688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:33.806947 30696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:33.806982 30694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:33.806936 30699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:33.807375 30688 server_base.cc:1034] running on GCE node
I20260807 08:37:33.807871 30688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:33.808643 30688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:33.809808 30688 hybrid_clock.cc:648] HybridClock initialized: now 1786091853809797 us; error 29 us; skew 500 ppm
Aug 07 08:37:33 dist-test-slave-h3ft krb5kdc[30130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:33.813280 30688 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:37:33.814301 30688 webserver.cc:460] Webserver started at http://127.9.174.3:36183/ using document root <none> and password file <none>
I20260807 08:37:33.814976 30688 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:33.815111 30688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:33.815364 30688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:33.817819 30688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "333284d47de5482393fdc69fdc47e8ce"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h3ft"
server_key: "8e14f7588dfdd1aed94a7ad1dcc7b04c"
server_key_iv: "3aeeb6c9d3a25177db00abe56c6e5821"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.818495 30688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "333284d47de5482393fdc69fdc47e8ce"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h3ft"
server_key: "8e14f7588dfdd1aed94a7ad1dcc7b04c"
server_key_iv: "3aeeb6c9d3a25177db00abe56c6e5821"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.822870 30688 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 08:37:33.825647 30706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.827167 30688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:33.827272 30688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "333284d47de5482393fdc69fdc47e8ce"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h3ft"
server_key: "8e14f7588dfdd1aed94a7ad1dcc7b04c"
server_key_iv: "3aeeb6c9d3a25177db00abe56c6e5821"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.827368 30688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:33.840108 30688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:33.844166 30688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:33.844393 30688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:33.845098 30688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:33.846107 30688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:33.846199 30688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.846290 30688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:33.846335 30688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.857086 30688 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:46317
I20260807 08:37:33.857136 30869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:46317 every 8 connection(s)
I20260807 08:37:33.858268 30688 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:37:33.864420 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 30688
I20260807 08:37:33.864614 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:37:33.864981 9912 external_mini_cluster.cc:1428] Setting key a43edd72a7d7fb84f36050fbf6ed9a66
Aug 07 08:37:33 dist-test-slave-h3ft krb5kdc[30130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:33.875733 30265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.860761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:44145 (local address 127.9.174.62:45261)
0807 08:37:33.860984 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:33.860988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:33.862035 (+ 1047us) server_negotiation.cc:407] Connection header received
0807 08:37:33.863726 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:33.863730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:33.863808 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:33.863926 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:33.865589 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.866488 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.868185 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.868441 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.868553 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.871510 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:33.871513 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:33.871516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:33.871543 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:33.873213 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.873821 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.873825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.873826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.873916 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.874309 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.874312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.874314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.874547 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:33.874663 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:33.875343 (+ 680us) server_negotiation.cc:299] Negotiation successful
0807 08:37:33.875508 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 08:37:33.876801 30872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.861206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:44145)
0807 08:37:33.861803 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:33.861858 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:33.863483 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:33.864159 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:33.864168 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:33.864716 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:33.865416 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.865438 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.866644 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.866649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.868054 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.868066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.868604 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.868607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.868766 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.869469 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:33.869488 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:33.871290 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:33.873350 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:33.873356 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:33.873377 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:33.873695 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:33.874021 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:33.874025 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:33.874027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:33.874197 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:33.874664 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:33.874672 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:33.874994 (+ 322us) client_negotiation.cc:769] Sending connection context
0807 08:37:33.875280 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 08:37:33.875688 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":169,"threads_started":1}
I20260807 08:37:33.878160 30870 heartbeater.cc:347] Connected to a master server at 127.9.174.62:45261
I20260807 08:37:33.878497 30870 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:33.879274 30870 heartbeater.cc:511] Master 127.9.174.62:45261 requested a full tablet report, sending...
I20260807 08:37:33.880491 30202 ts_manager.cc:194] Registered new tserver with Master: 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317)
I20260807 08:37:33.881167 30202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:44145
I20260807 08:37:33.891480 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:37:33.902160 30265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.894054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41832 (local address 127.9.174.62:45261)
0807 08:37:33.894249 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:33.894254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:33.894401 (+ 147us) server_negotiation.cc:407] Connection header received
0807 08:37:33.894592 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:33.894595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:33.894646 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:33.894732 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:33.895609 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.896199 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.897103 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.897333 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.897448 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.898932 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:33.898936 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:33.898938 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:33.898965 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:33.900319 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.900768 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.900773 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.900775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.900866 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.901149 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.901153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.901155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.901423 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:33.901590 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:33.901827 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 08:37:33.901960 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 08:37:33.905488 30202 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:33.909830 30202 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:33.928735 30896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.922378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47996 (local address 127.9.174.3:46317)
0807 08:37:33.923176 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:33.923189 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:33.923212 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:37:33.923297 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:33.923303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:33.923495 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:33.923649 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:33.924073 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.925314 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.926790 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.927690 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.927819 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.928018 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:33.928399 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 08:37:33.928514 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":149,"threads_started":1}
I20260807 08:37:33.929064 30894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.921563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41440 (local address 127.9.174.2:41893)
0807 08:37:33.922088 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:33.922106 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:33.922641 (+ 535us) server_negotiation.cc:407] Connection header received
0807 08:37:33.923017 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:33.923025 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:33.923282 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:33.923448 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:33.923940 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.925314 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.927191 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.928040 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.928149 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.928309 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:33.928638 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 08:37:33.928744 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":258,"threads_started":1}
I20260807 08:37:33.928843 30892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.921339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:41893 (local address 127.0.0.1:41440)
0807 08:37:33.922437 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:33.922523 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:33.922731 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:33.923554 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:33.923561 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:33.923589 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:33.923795 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.923804 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.925447 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.925450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.927057 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.927069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.928382 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.928387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.928484 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.928508 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 08:37:33.928586 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 08:37:33.928669 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":868,"mutex_wait_us":227,"thread_start_us":109,"threads_started":1}
I20260807 08:37:33.929442 30891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.921381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:46317 (local address 127.0.0.1:47996)
0807 08:37:33.922439 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:33.922517 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:33.922674 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:33.923690 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:33.923694 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:33.923714 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:33.923900 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.923908 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.925495 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.925498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.926640 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.926650 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.927824 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.927828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.927949 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.928044 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 08:37:33.928171 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 08:37:33.929028 (+ 857us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"mutex_wait_us":145,"thread_start_us":156,"threads_started":1}
I20260807 08:37:33.929570 30900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.922613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:42079 (local address 127.0.0.1:46400)
0807 08:37:33.923064 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:33.923079 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:33.923177 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:33.923971 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:33.923974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:33.923999 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:33.924080 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.924086 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.925397 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.925404 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.927057 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.927069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.929107 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.929110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.929207 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.929237 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 08:37:33.929289 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 08:37:33.929372 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":175,"thread_start_us":107,"threads_started":1}
I20260807 08:37:33.930439 30901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.923117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46400 (local address 127.9.174.1:42079)
0807 08:37:33.923520 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:33.923527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:33.923548 (+ 21us) server_negotiation.cc:407] Connection header received
0807 08:37:33.923627 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:33.923632 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:33.923814 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:33.923999 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:33.924167 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.925283 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.928106 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.928967 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.929946 (+ 979us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.930110 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:33.930163 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 08:37:33.930272 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":182,"threads_started":1}
I20260807 08:37:33.933269 30387 tablet_service.cc:1467] Processing CreateTablet for tablet 3d89ccb26cf04e69835ba1825e3f0d1d (DEFAULT_TABLE table=test-table [id=000372d8651f46b9a18ccc0b9e096a78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:33.933282 30776 tablet_service.cc:1467] Processing CreateTablet for tablet 3d89ccb26cf04e69835ba1825e3f0d1d (DEFAULT_TABLE table=test-table [id=000372d8651f46b9a18ccc0b9e096a78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:33.933282 30580 tablet_service.cc:1467] Processing CreateTablet for tablet 3d89ccb26cf04e69835ba1825e3f0d1d (DEFAULT_TABLE table=test-table [id=000372d8651f46b9a18ccc0b9e096a78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:33.934649 30387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d89ccb26cf04e69835ba1825e3f0d1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:33.934653 30580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d89ccb26cf04e69835ba1825e3f0d1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:33.934667 30776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d89ccb26cf04e69835ba1825e3f0d1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:33.943917 30903 tablet_bootstrap.cc:492] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: Bootstrap starting.
I20260807 08:37:33.943908 30904 tablet_bootstrap.cc:492] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a: Bootstrap starting.
I20260807 08:37:33.944815 30907 tablet_bootstrap.cc:492] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce: Bootstrap starting.
I20260807 08:37:33.946609 30903 tablet_bootstrap.cc:654] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:33.947130 30904 tablet_bootstrap.cc:654] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:33.947413 30907 tablet_bootstrap.cc:654] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:33.947647 30903 log.cc:824] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:33.948086 30904 log.cc:824] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:33.948240 30907 log.cc:824] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:33.949889 30903 tablet_bootstrap.cc:492] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: No bootstrap required, opened a new log
I20260807 08:37:33.949934 30907 tablet_bootstrap.cc:492] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce: No bootstrap required, opened a new log
I20260807 08:37:33.950130 30903 ts_tablet_manager.cc:1397] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260807 08:37:33.950130 30907 ts_tablet_manager.cc:1397] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 08:37:33.952872 30904 tablet_bootstrap.cc:492] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a: No bootstrap required, opened a new log
I20260807 08:37:33.953107 30904 ts_tablet_manager.cc:1397] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260807 08:37:33.954342 30903 raft_consensus.cc:348] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:33.954638 30903 raft_consensus.cc:374] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:33.954694 30903 raft_consensus.cc:729] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94499e8e1b714946bf6d45bcfeafdb86, State: Initialized, Role: FOLLOWER
I20260807 08:37:33.954485 30907 raft_consensus.cc:348] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:33.954761 30907 raft_consensus.cc:374] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:33.954814 30907 raft_consensus.cc:729] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 333284d47de5482393fdc69fdc47e8ce, State: Initialized, Role: FOLLOWER
I20260807 08:37:33.955330 30903 consensus_queue.cc:260] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:33.955332 30907 consensus_queue.cc:260] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:33.956403 30903 ts_tablet_manager.cc:1428] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260807 08:37:33.956422 30907 ts_tablet_manager.cc:1428] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 08:37:33.956876 30870 heartbeater.cc:503] Master 127.9.174.62:45261 was elected leader, sending a full tablet report...
I20260807 08:37:33.956876 30487 heartbeater.cc:503] Master 127.9.174.62:45261 was elected leader, sending a full tablet report...
I20260807 08:37:33.957337 30904 raft_consensus.cc:348] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:33.957563 30904 raft_consensus.cc:374] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:33.957607 30904 raft_consensus.cc:729] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83d9331d2ca54303a1f19c963d20017a, State: Initialized, Role: FOLLOWER
I20260807 08:37:33.958042 30904 consensus_queue.cc:260] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:33.958807 30679 heartbeater.cc:503] Master 127.9.174.62:45261 was elected leader, sending a full tablet report...
I20260807 08:37:33.959226 30904 ts_tablet_manager.cc:1428] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260807 08:37:34.094292 30488 tablet.cc:2369] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:37:34.101454 30680 tablet.cc:2369] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:37:34.109599 30871 tablet.cc:2369] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:37:34.157087 30917 raft_consensus.cc:482] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:34.157483 30917 raft_consensus.cc:504] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:34.159279 30917 leader_election.cc:288] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83d9331d2ca54303a1f19c963d20017a (127.9.174.2:41893), 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317)
I20260807 08:37:34.164108 30894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:34.159736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:53135 (local address 127.9.174.2:41893)
0807 08:37:34.159915 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:34.159923 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:34.160326 (+ 403us) server_negotiation.cc:407] Connection header received
0807 08:37:34.160487 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:34.160490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:34.160564 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:34.160652 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:34.161054 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.161571 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.162882 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.163399 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.163535 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:34.163626 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:34.163846 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 08:37:34.163949 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 08:37:34.164206 30921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:34.159615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:46317 (local address 127.9.174.1:44487)
0807 08:37:34.160106 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:34.160121 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:34.160239 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:34.160588 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:34.160593 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:34.160615 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:34.160822 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:34.160829 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.161737 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.161740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:34.162716 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:34.162726 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.163625 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.163628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:34.163771 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:34.163814 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 08:37:34.163930 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 08:37:34.163998 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":105,"thread_start_us":168,"threads_started":1}
I20260807 08:37:34.164203 30896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:34.159793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:44487 (local address 127.9.174.3:46317)
0807 08:37:34.160015 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:34.160021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:34.160219 (+ 198us) server_negotiation.cc:407] Connection header received
0807 08:37:34.160393 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:34.160400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:34.160489 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:34.160558 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:34.160963 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.161595 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.162848 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.163433 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.163570 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:34.163651 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:34.163962 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 08:37:34.164057 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260807 08:37:34.164431 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:34.159618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:41893 (local address 127.9.174.1:53135)
0807 08:37:34.160161 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:34.160195 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:34.160350 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:34.160687 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:34.160691 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:34.160709 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:34.160920 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:34.160931 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.161696 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.161701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:34.162725 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:34.162735 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.163533 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.163539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:34.163701 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:34.163730 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 08:37:34.163829 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 08:37:34.163918 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":117,"thread_start_us":131,"threads_started":1}
I20260807 08:37:34.165020 30805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d89ccb26cf04e69835ba1825e3f0d1d" candidate_uuid: "94499e8e1b714946bf6d45bcfeafdb86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "333284d47de5482393fdc69fdc47e8ce" is_pre_election: true
I20260807 08:37:34.165165 30612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d89ccb26cf04e69835ba1825e3f0d1d" candidate_uuid: "94499e8e1b714946bf6d45bcfeafdb86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83d9331d2ca54303a1f19c963d20017a" is_pre_election: true
I20260807 08:37:34.165469 30805 raft_consensus.cc:2454] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94499e8e1b714946bf6d45bcfeafdb86 in term 0.
I20260807 08:37:34.165509 30612 raft_consensus.cc:2454] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94499e8e1b714946bf6d45bcfeafdb86 in term 0.
I20260807 08:37:34.166023 30319 leader_election.cc:302] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 333284d47de5482393fdc69fdc47e8ce, 94499e8e1b714946bf6d45bcfeafdb86; no voters:
I20260807 08:37:34.166339 30917 raft_consensus.cc:2781] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:34.166435 30917 raft_consensus.cc:482] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:34.166479 30917 raft_consensus.cc:3037] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:34.167878 30917 raft_consensus.cc:504] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:34.168288 30917 leader_election.cc:288] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 1 election: Requested vote from peers 83d9331d2ca54303a1f19c963d20017a (127.9.174.2:41893), 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317)
I20260807 08:37:34.168731 30612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d89ccb26cf04e69835ba1825e3f0d1d" candidate_uuid: "94499e8e1b714946bf6d45bcfeafdb86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83d9331d2ca54303a1f19c963d20017a"
I20260807 08:37:34.168759 30805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d89ccb26cf04e69835ba1825e3f0d1d" candidate_uuid: "94499e8e1b714946bf6d45bcfeafdb86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "333284d47de5482393fdc69fdc47e8ce"
I20260807 08:37:34.168893 30612 raft_consensus.cc:3037] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:34.168905 30805 raft_consensus.cc:3037] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:34.170199 30612 raft_consensus.cc:2454] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94499e8e1b714946bf6d45bcfeafdb86 in term 1.
I20260807 08:37:34.170204 30805 raft_consensus.cc:2454] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94499e8e1b714946bf6d45bcfeafdb86 in term 1.
I20260807 08:37:34.170562 30319 leader_election.cc:302] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 333284d47de5482393fdc69fdc47e8ce, 94499e8e1b714946bf6d45bcfeafdb86; no voters:
I20260807 08:37:34.170758 30917 raft_consensus.cc:2781] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:34.171097 30917 raft_consensus.cc:686] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 LEADER]: Becoming Leader. State: Replica: 94499e8e1b714946bf6d45bcfeafdb86, State: Running, Role: LEADER
I20260807 08:37:34.171566 30917 consensus_queue.cc:237] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:34.175763 30202 catalog_manager.cc:5495] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94499e8e1b714946bf6d45bcfeafdb86 (127.9.174.1). New cstate: current_term: 1 leader_uuid: "94499e8e1b714946bf6d45bcfeafdb86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } health_report { overall_health: HEALTHY } } }
I20260807 08:37:34.226193 30901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:34.221994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46414 (local address 127.9.174.1:42079)
0807 08:37:34.222147 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:34.222151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:34.222167 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:37:34.222319 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:34.222322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:34.222395 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:34.222512 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:34.222749 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.223389 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.224385 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.224620 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.224707 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:34.225101 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:34.225706 (+ 605us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:34.225856 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:34.225968 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 08:37:34.226041 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 08:37:34.233621 30612 raft_consensus.cc:1261] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a [term 1 FOLLOWER]: Refusing update from remote peer 94499e8e1b714946bf6d45bcfeafdb86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:34.233621 30805 raft_consensus.cc:1261] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce [term 1 FOLLOWER]: Refusing update from remote peer 94499e8e1b714946bf6d45bcfeafdb86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:34.234540 30923 consensus_queue.cc:1035] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:34.234755 30917 consensus_queue.cc:1035] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:34.243741 30927 mvcc.cc:205] Tried to move back new op lower bound from 7315832234931757056 to 7315832234690052096. Current Snapshot: MvccSnapshot[applied={T|T < 7315832234931757056}]
I20260807 08:37:34.253096 30202 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:34.253304 30202 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:34.256270 30202 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: Sending DeleteTablet for 3 replicas of tablet 3d89ccb26cf04e69835ba1825e3f0d1d
I20260807 08:37:34.257256 30580 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d89ccb26cf04e69835ba1825e3f0d1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:34 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:41440
I20260807 08:37:34.257397 30387 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d89ccb26cf04e69835ba1825e3f0d1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:34 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:46400
I20260807 08:37:34.257530 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 30297
I20260807 08:37:34.257526 30776 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d89ccb26cf04e69835ba1825e3f0d1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:34 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:47996
I20260807 08:37:34.265849 30950 tablet_replica.cc:330] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a: stopping tablet replica
I20260807 08:37:34.266491 30950 raft_consensus.cc:2229] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:34.266965 30950 raft_consensus.cc:2258] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:34.269255 30953 tablet_replica.cc:330] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce: stopping tablet replica
I20260807 08:37:34.269591 30953 raft_consensus.cc:2229] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:34.269923 30953 raft_consensus.cc:2258] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:34.270140 30950 ts_tablet_manager.cc:1905] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:34.271310 30953 ts_tablet_manager.cc:1905] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:34.273762 30953 ts_tablet_manager.cc:1918] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:34.273890 30953 log.cc:1196] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/3d89ccb26cf04e69835ba1825e3f0d1d
I20260807 08:37:34.274222 30953 ts_tablet_manager.cc:1939] T 3d89ccb26cf04e69835ba1825e3f0d1d P 333284d47de5482393fdc69fdc47e8ce: Deleting consensus metadata
I20260807 08:37:34.274473 30950 ts_tablet_manager.cc:1918] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:34.274591 30950 log.cc:1196] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/3d89ccb26cf04e69835ba1825e3f0d1d
I20260807 08:37:34.274941 30950 ts_tablet_manager.cc:1939] T 3d89ccb26cf04e69835ba1825e3f0d1d P 83d9331d2ca54303a1f19c963d20017a: Deleting consensus metadata
I20260807 08:37:34.275815 30177 catalog_manager.cc:4841] TS 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317): tablet 3d89ccb26cf04e69835ba1825e3f0d1d (table test-table [id=000372d8651f46b9a18ccc0b9e096a78]) successfully deleted
I20260807 08:37:34.276294 30175 catalog_manager.cc:4841] TS 83d9331d2ca54303a1f19c963d20017a (127.9.174.2:41893): tablet 3d89ccb26cf04e69835ba1825e3f0d1d (table test-table [id=000372d8651f46b9a18ccc0b9e096a78]) successfully deleted
W20260807 08:37:34.279274 30180 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:42079 (error 108)
I20260807 08:37:34.279686 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 30495
I20260807 08:37:34.288319 30892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:34.287858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:42079 (local address 127.0.0.1:46416)
0807 08:37:34.288059 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:34.288168 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:42079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 08:37:34.288578 30180 catalog_manager.cc:4584] TS 94499e8e1b714946bf6d45bcfeafdb86 (127.9.174.1:42079): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3d89ccb26cf04e69835ba1825e3f0d1d: Network error: Client connection negotiation failed: client connection to 127.9.174.1:42079: connect: Connection refused (error 111)
I20260807 08:37:34.295627 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 30688
I20260807 08:37:34.308589 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 30147
I20260807 08:37:34.322239 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:45261
--webserver_interface=127.9.174.62
--webserver_port=38961
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:45261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:34.455797 30955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:34.456133 30955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:34.456180 30955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:34.460398 30955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:34.460474 30955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:34.460496 30955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:34.460553 30955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:34.460588 30955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:34.466169 30955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:45261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:45261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=38961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:34.467545 30955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:34.468879 30955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:34.485942 30971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:34.486184 30955 server_base.cc:1034] running on GCE node
W20260807 08:37:34.485942 30974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:34.485996 30972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:34.486984 30955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:34.488811 30955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:34.490021 30955 hybrid_clock.cc:648] HybridClock initialized: now 1786091854489985 us; error 48 us; skew 500 ppm
Aug 07 08:37:34 dist-test-slave-h3ft krb5kdc[30130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091854, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:34.493808 30955 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:37:34.495083 30955 webserver.cc:460] Webserver started at http://127.9.174.62:38961/ using document root <none> and password file <none>
I20260807 08:37:34.495806 30955 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:34.495925 30955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:34.500675 30955 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:37:34.503569 30982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:34.505152 30955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:34.505316 30955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "011c5d96f7874fe5b6bf66fe2a7adc11"
format_stamp: "Formatted at 2026-08-07 08:37:32 on dist-test-slave-h3ft"
server_key: "81b24a94c54ce3c4ace2b179c39d4f1e"
server_key_iv: "de7bd4b8e2a46e462af8728c63a7f38f"
server_key_version: "encryptionkey@0"
I20260807 08:37:34.506074 30955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:34.533156 30955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:34.537042 30955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:34.537298 30955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:34.546198 30955 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:45261
I20260807 08:37:34.546518 31064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:45261 every 8 connection(s)
I20260807 08:37:34.547572 30955 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:34.551288 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 30955
I20260807 08:37:34.551779 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:42079
--local_ip_for_outbound_sockets=127.9.174.1
--tserver_master_addrs=127.9.174.62:45261
--webserver_port=44597
--webserver_interface=127.9.174.1
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:37:34.553004 31065 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:37:34.554877 31065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: Bootstrap starting.
I20260807 08:37:34.576478 31065 log.cc:824] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:34.586457 31065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:37:34.586903 31065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: Bootstrap complete.
I20260807 08:37:34.591733 31065 raft_consensus.cc:348] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } }
I20260807 08:37:34.592186 31065 raft_consensus.cc:729] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011c5d96f7874fe5b6bf66fe2a7adc11, State: Initialized, Role: FOLLOWER
I20260807 08:37:34.592779 31065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } }
I20260807 08:37:34.592933 31065 raft_consensus.cc:388] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:34.593034 31065 raft_consensus.cc:482] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:34.593155 31065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:37:34.595301 31065 raft_consensus.cc:504] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } }
I20260807 08:37:34.595670 31065 leader_election.cc:302] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 011c5d96f7874fe5b6bf66fe2a7adc11; no voters:
I20260807 08:37:34.596024 31065 leader_election.cc:288] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 08:37:34.596177 31069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:37:34.596462 31069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 2 LEADER]: Becoming Leader. State: Replica: 011c5d96f7874fe5b6bf66fe2a7adc11, State: Running, Role: LEADER
I20260807 08:37:34.596804 31069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } }
I20260807 08:37:34.597393 31065 sys_catalog.cc:564] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:34.598541 31072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 011c5d96f7874fe5b6bf66fe2a7adc11. Latest consensus state: current_term: 2 leader_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } } }
I20260807 08:37:34.598570 31070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } } }
I20260807 08:37:34.598711 31070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:34.598709 31072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:34.599283 31080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:34.605049 31080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:34.606132 31080 catalog_manager.cc:1259] Loaded cluster ID: 7d4fb95049ee411183dfb9b75bea2c88
I20260807 08:37:34.606204 31080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:34.607863 31080 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 08:37:34.613411 31080 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 08:37:34.620182 31080 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:34.621155 31080 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: Loaded TSK: 0
I20260807 08:37:34.622609 31080 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:37:34.700135 31067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:34.700515 31067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:34.700593 31067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:34.705008 31067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:34.705121 31067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:34.705219 31067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:37:34.711678 31067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:42079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=44597
--tserver_master_addrs=127.9.174.62:45261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:34.713109 31067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:34.714434 31067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:34.726758 31099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:34.726758 31100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:34.727293 31102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:34.727341 31067 server_base.cc:1034] running on GCE node
I20260807 08:37:34.727910 31067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:34.728797 31067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:34.729996 31067 hybrid_clock.cc:648] HybridClock initialized: now 1786091854729969 us; error 44 us; skew 500 ppm
Aug 07 08:37:34 dist-test-slave-h3ft krb5kdc[30130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091854, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:34.737804 31067 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:37:34.739352 31067 webserver.cc:460] Webserver started at http://127.9.174.1:44597/ using document root <none> and password file <none>
I20260807 08:37:34.740268 31067 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:34.740418 31067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:34.746673 31067 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 08:37:34.749696 31113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:34.751514 31067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 08:37:34.751659 31067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "94499e8e1b714946bf6d45bcfeafdb86"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h3ft"
server_key: "288fa9ba02a1ab11d040bca02c663fb2"
server_key_iv: "a8299ab9e4afec4df99a0c0aab22195c"
server_key_version: "encryptionkey@0"
I20260807 08:37:34.752532 31067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:34.774288 31067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:34.778730 31067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:34.778976 31067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:34.779879 31067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:34.781419 31122 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:37:34.786967 31067 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:37:34.787081 31067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260807 08:37:34.787146 31067 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:37:34.789247 31067 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:37:34.789333 31067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 08:37:34.789656 31122 tablet_bootstrap.cc:492] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: Bootstrap starting.
I20260807 08:37:34.802807 31067 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:42079
I20260807 08:37:34.802829 31291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:42079 every 8 connection(s)
I20260807 08:37:34.804062 31067 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:34.812057 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 31067
I20260807 08:37:34.812636 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:41893
--local_ip_for_outbound_sockets=127.9.174.2
--tserver_master_addrs=127.9.174.62:45261
--webserver_port=44541
--webserver_interface=127.9.174.2
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:37:34.813934 31122 log.cc:824] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:37:34 dist-test-slave-h3ft krb5kdc[30130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091854, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:34.824721 31122 tablet_bootstrap.cc:492] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:37:34.825196 31122 tablet_bootstrap.cc:492] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: Bootstrap complete.
I20260807 08:37:34.827162 31122 ts_tablet_manager.cc:1397] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: Time spent bootstrapping tablet: real 0.038s user 0.019s sys 0.017s
I20260807 08:37:34.827807 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:34.806567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:48853)
0807 08:37:34.807746 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:34.807818 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:34.809291 (+ 1473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:34.811061 (+ 1770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:34.811079 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:34.811794 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:34.812544 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:34.812568 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.814699 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.814703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:34.815817 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:34.815828 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.816891 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.816898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:34.817139 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:34.817982 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:34.818010 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:34.820268 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:34.823531 (+ 3263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:34.823537 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:34.823549 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:34.823918 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:34.824424 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:34.824430 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:34.824434 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:34.824624 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:34.825530 (+ 906us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:34.825539 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:34.825883 (+ 344us) client_negotiation.cc:769] Sending connection context
0807 08:37:34.826091 (+ 208us) client_negotiation.cc:240] Negotiation successful
0807 08:37:34.826524 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"thread_start_us":206,"threads_started":1}
I20260807 08:37:34.827812 31296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:34.806643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:48853 (local address 127.9.174.62:45261)
0807 08:37:34.807234 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:34.807265 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:34.807992 (+ 727us) server_negotiation.cc:407] Connection header received
0807 08:37:34.809876 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:34.809891 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:34.810465 (+ 574us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:34.810794 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:34.812799 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.814494 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.815963 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.816704 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.816903 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:34.820561 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:34.820583 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:34.820605 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:34.820655 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:34.823367 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:34.824079 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:34.824089 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:34.824097 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:34.824299 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:34.824756 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:34.824762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:34.824765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:34.825389 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:34.825560 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:34.826287 (+ 727us) server_negotiation.cc:299] Negotiation successful
0807 08:37:34.826698 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":181,"threads_started":1}
I20260807 08:37:34.830492 31122 raft_consensus.cc:348] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:34.831092 31122 raft_consensus.cc:729] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94499e8e1b714946bf6d45bcfeafdb86, State: Initialized, Role: FOLLOWER
I20260807 08:37:34.831353 31292 heartbeater.cc:347] Connected to a master server at 127.9.174.62:45261
I20260807 08:37:34.831634 31292 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:34.831749 31122 consensus_queue.cc:260] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:34.832329 31292 heartbeater.cc:511] Master 127.9.174.62:45261 requested a full tablet report, sending...
I20260807 08:37:34.833001 31122 ts_tablet_manager.cc:1428] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
I20260807 08:37:34.835834 31012 ts_manager.cc:194] Registered new tserver with Master: 94499e8e1b714946bf6d45bcfeafdb86 (127.9.174.1:42079)
I20260807 08:37:34.840806 31012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:48853
I20260807 08:37:34.845429 31292 heartbeater.cc:503] Master 127.9.174.62:45261 was elected leader, sending a full tablet report...
Aug 07 08:37:34 dist-test-slave-h3ft krb5kdc[30130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091854, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for kudu/127.9.174.1@KRBTEST.COM
I20260807 08:37:34.855067 31308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:34.840233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46432 (local address 127.9.174.1:42079)
0807 08:37:34.840774 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:34.840787 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:34.841265 (+ 478us) server_negotiation.cc:407] Connection header received
0807 08:37:34.841523 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:34.841528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:34.841714 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:34.841879 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:34.842883 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.844081 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.845357 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.845870 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.845987 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:34.849564 (+ 3577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:34.849577 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:34.849592 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:34.849620 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:34.852807 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:34.853450 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:34.853457 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:34.853459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:34.853560 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:34.853857 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:34.853861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:34.853863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:34.854163 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:34.854316 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:34.854675 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 08:37:34.854866 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":423,"thread_start_us":149,"threads_started":1}
I20260807 08:37:34.855067 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:34.840132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:42079 (local address 127.0.0.1:46432)
0807 08:37:34.841106 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:34.841151 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:34.841341 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:34.842008 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:34.842017 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:34.842561 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:34.842726 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:34.842736 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.844229 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.844233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:34.845214 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:34.845226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:34.846132 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:34.846136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:34.846267 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:34.847179 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:34.847199 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:34.849399 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:34.852921 (+ 3522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:34.852931 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:34.852937 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:34.853282 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:34.853659 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:34.853664 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:34.853666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:34.853764 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:34.854283 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:34.854289 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:34.854431 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 08:37:34.854576 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 08:37:34.854813 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":128,"threads_started":1}
I20260807 08:37:34.856289 31181 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d89ccb26cf04e69835ba1825e3f0d1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:34 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:46432
I20260807 08:37:34.857532 31309 tablet_replica.cc:330] stopping tablet replica
I20260807 08:37:34.857918 31309 raft_consensus.cc:2229] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:34.858278 31309 raft_consensus.cc:2258] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:34.859733 31309 ts_tablet_manager.cc:1905] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:34.864212 31309 ts_tablet_manager.cc:1918] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:34.864351 31309 log.cc:1196] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal/wals/3d89ccb26cf04e69835ba1825e3f0d1d
I20260807 08:37:34.864698 31309 ts_tablet_manager.cc:1939] T 3d89ccb26cf04e69835ba1825e3f0d1d P 94499e8e1b714946bf6d45bcfeafdb86: Deleting consensus metadata
I20260807 08:37:34.866811 30988 catalog_manager.cc:4841] TS 94499e8e1b714946bf6d45bcfeafdb86 (127.9.174.1:42079): tablet 3d89ccb26cf04e69835ba1825e3f0d1d (table test-table [id=000372d8651f46b9a18ccc0b9e096a78]) successfully deleted
W20260807 08:37:34.959112 31300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:34.959510 31300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:34.959618 31300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:34.963872 31300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:34.963943 31300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:34.964036 31300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:37:34.969762 31300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:41893
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=44541
--tserver_master_addrs=127.9.174.62:45261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:34.971239 31300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:34.972597 31300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:34.984597 31321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:34.984608 31316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:34.984629 31326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:34.985059 31300 server_base.cc:1034] running on GCE node
I20260807 08:37:34.985575 31300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:34.986366 31300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:34.987560 31300 hybrid_clock.cc:648] HybridClock initialized: now 1786091854987542 us; error 29 us; skew 500 ppm
Aug 07 08:37:34 dist-test-slave-h3ft krb5kdc[30130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091854, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:34.990664 31300 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:37:34.991791 31300 webserver.cc:460] Webserver started at http://127.9.174.2:44541/ using document root <none> and password file <none>
I20260807 08:37:34.992489 31300 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:34.992612 31300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:34.997211 31300 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:37:34.999766 31334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:35.001269 31300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:35.001391 31300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "83d9331d2ca54303a1f19c963d20017a"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h3ft"
server_key: "36c65ec9cc60e4d23b538bb9ac9ea972"
server_key_iv: "56f38d67babe411cb206a94cbdf76fd9"
server_key_version: "encryptionkey@0"
I20260807 08:37:35.002113 31300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:35.034667 31300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:35.038484 31300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:35.038739 31300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:35.039578 31300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:35.040706 31300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:35.040786 31300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:35.040874 31300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:35.040931 31300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:35.052878 31300 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:41893
I20260807 08:37:35.052901 31506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:41893 every 8 connection(s)
I20260807 08:37:35.054260 31300 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:35.063628 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 31300
I20260807 08:37:35.064210 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:46317
--local_ip_for_outbound_sockets=127.9.174.3
--tserver_master_addrs=127.9.174.62:45261
--webserver_port=36183
--webserver_interface=127.9.174.3
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:37:35 dist-test-slave-h3ft krb5kdc[30130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091854, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:35.071134 31296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.056614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:41183 (local address 127.9.174.62:45261)
0807 08:37:35.056842 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:35.056848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:35.057845 (+ 997us) server_negotiation.cc:407] Connection header received
0807 08:37:35.059535 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:35.059538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:35.059607 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:35.059695 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:35.061077 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.061707 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.062939 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.063355 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.063500 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.066561 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:35.066568 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:35.066576 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:35.066623 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:35.068362 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:35.069053 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:35.069061 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:35.069063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:35.069172 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:35.069596 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:35.069599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:35.069600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:35.069814 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:35.069898 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:35.070658 (+ 760us) server_negotiation.cc:299] Negotiation successful
0807 08:37:35.070818 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 08:37:35.072614 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.057125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:41183)
0807 08:37:35.057665 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:35.057713 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:35.059307 (+ 1594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:35.059887 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:35.059894 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:35.060341 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:35.060915 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:35.060928 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.061822 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.061824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:35.062801 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:35.062811 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.063525 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.063528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:35.063696 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.064568 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:35.064588 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:35.066340 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:35.068496 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:35.068505 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:35.068526 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:35.068919 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:35.069277 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:35.069283 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:35.069287 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:35.069495 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:35.069922 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:35.069932 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:35.070334 (+ 402us) client_negotiation.cc:769] Sending connection context
0807 08:37:35.070620 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 08:37:35.071130 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":150,"threads_started":1}
I20260807 08:37:35.074263 31512 heartbeater.cc:347] Connected to a master server at 127.9.174.62:45261
I20260807 08:37:35.074585 31512 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:35.075285 31512 heartbeater.cc:511] Master 127.9.174.62:45261 requested a full tablet report, sending...
I20260807 08:37:35.076681 31012 ts_manager.cc:194] Registered new tserver with Master: 83d9331d2ca54303a1f19c963d20017a (127.9.174.2:41893)
I20260807 08:37:35.077366 31012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:41183
W20260807 08:37:35.203197 31516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:35.203615 31516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:35.203698 31516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:35.207988 31516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:35.208053 31516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:35.208141 31516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:37:35.214051 31516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:46317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=36183
--tserver_master_addrs=127.9.174.62:45261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:35.215477 31516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:35.216791 31516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:35.228634 31523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:35.228626 31521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:35.228819 31516 server_base.cc:1034] running on GCE node
W20260807 08:37:35.228626 31525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:35.229542 31516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:35.230336 31516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:35.231504 31516 hybrid_clock.cc:648] HybridClock initialized: now 1786091855231489 us; error 34 us; skew 500 ppm
Aug 07 08:37:35 dist-test-slave-h3ft krb5kdc[30130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091855, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:35.234864 31516 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:37:35.235987 31516 webserver.cc:460] Webserver started at http://127.9.174.3:36183/ using document root <none> and password file <none>
I20260807 08:37:35.236701 31516 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:35.236824 31516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:35.241496 31516 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:37:35.244118 31537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:35.245667 31516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:35.245834 31516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "333284d47de5482393fdc69fdc47e8ce"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h3ft"
server_key: "8e14f7588dfdd1aed94a7ad1dcc7b04c"
server_key_iv: "3aeeb6c9d3a25177db00abe56c6e5821"
server_key_version: "encryptionkey@0"
I20260807 08:37:35.246579 31516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:35.267915 31516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:35.272083 31516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:35.272342 31516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:35.273200 31516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:35.274319 31516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:35.274407 31516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:35.274507 31516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:35.274565 31516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:35.285552 31516 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:46317
I20260807 08:37:35.285710 31710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:46317 every 8 connection(s)
I20260807 08:37:35.286882 31516 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:37:35.295204 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 31516
Aug 07 08:37:35 dist-test-slave-h3ft krb5kdc[30130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091855, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:35.304193 31296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.289297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:43903 (local address 127.9.174.62:45261)
0807 08:37:35.289472 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:35.289477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:35.290511 (+ 1034us) server_negotiation.cc:407] Connection header received
0807 08:37:35.292096 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:35.292100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:35.292171 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:35.292275 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:35.293709 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.294306 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.295682 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.295996 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.296114 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.299402 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:35.299407 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:35.299410 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:35.299439 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:35.301341 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:35.302088 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:35.302091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:35.302093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:35.302201 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:35.302672 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:35.302675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:35.302676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:35.302928 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:35.303065 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:35.303769 (+ 704us) server_negotiation.cc:299] Negotiation successful
0807 08:37:35.303928 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 08:37:35.305451 31713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.289787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:43903)
0807 08:37:35.290327 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:35.290375 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:35.291851 (+ 1476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:35.292486 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:35.292494 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:35.292959 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:35.293543 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:35.293559 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.294447 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.294450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:35.295555 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:35.295566 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.296167 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.296171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:35.296364 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.297130 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:35.297151 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:35.299169 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:35.301477 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:35.301483 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:35.301502 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:35.301939 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:35.302330 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:35.302336 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:35.302339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:35.302547 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:35.303068 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:35.303079 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:35.303420 (+ 341us) client_negotiation.cc:769] Sending connection context
0807 08:37:35.303666 (+ 246us) client_negotiation.cc:240] Negotiation successful
0807 08:37:35.304016 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":143,"threads_started":1}
I20260807 08:37:35.306905 31711 heartbeater.cc:347] Connected to a master server at 127.9.174.62:45261
I20260807 08:37:35.307243 31711 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:35.307247 31717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.296045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41840 (local address 127.9.174.62:45261)
0807 08:37:35.296427 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:35.296434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:35.296529 (+ 95us) server_negotiation.cc:407] Connection header received
0807 08:37:35.296725 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:35.296727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:35.296774 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:35.296853 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:35.297766 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.298469 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.299995 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.300412 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.300535 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.303149 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:35.303152 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:35.303154 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:35.303174 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:35.304862 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:35.305347 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:35.305351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:35.305353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:35.305466 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:35.306172 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:35.306176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:35.306178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:35.306397 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:35.306664 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:35.306884 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 08:37:35.307047 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":112,"threads_started":1}
I20260807 08:37:35.307902 31711 heartbeater.cc:511] Master 127.9.174.62:45261 requested a full tablet report, sending...
I20260807 08:37:35.309244 31011 ts_manager.cc:194] Registered new tserver with Master: 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317)
I20260807 08:37:35.309967 31011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:43903
I20260807 08:37:35.310518 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:37:35.315069 31717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.311274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41850 (local address 127.9.174.62:45261)
0807 08:37:35.311422 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:35.311427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:35.311656 (+ 229us) server_negotiation.cc:407] Connection header received
0807 08:37:35.311805 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:35.311807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:35.311915 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:35.311998 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:35.312247 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.312773 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.313624 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.313854 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.313958 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.314241 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:35.314658 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:35.314751 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:35.314802 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 08:37:35.314856 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:37:35.315800 31011 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:41850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:35.319886 31011 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:35.330135 31181 tablet_service.cc:1467] Processing CreateTablet for tablet e808904c387c4d088f4942c47d853052 (DEFAULT_TABLE table=test-table [id=0acf8c7d2b56497bae9ec33c9f1b3a3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:35.331092 31181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e808904c387c4d088f4942c47d853052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:35.334812 31726 tablet_bootstrap.cc:492] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86: Bootstrap starting.
I20260807 08:37:35.335970 31726 tablet_bootstrap.cc:654] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:35.336547 31723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.328563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:41893 (local address 127.0.0.1:41456)
0807 08:37:35.328925 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:35.328942 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:35.329054 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:35.330333 (+ 1279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:35.330336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:35.330357 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:35.330466 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:35.330472 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.332012 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.332015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:35.333364 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:35.333375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.335952 (+ 2577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.335965 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:35.336076 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.336151 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 08:37:35.336246 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 08:37:35.336365 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":29,"thread_start_us":188,"threads_started":1}
I20260807 08:37:35.336670 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.328262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:46317 (local address 127.0.0.1:48006)
0807 08:37:35.328414 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:35.328434 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:35.328537 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:35.330232 (+ 1695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:35.330235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:35.330256 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:35.330403 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:35.330414 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.331866 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.331873 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:35.333819 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:35.333830 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.336190 (+ 2360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.336194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:35.336324 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.336350 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 08:37:35.336403 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 08:37:35.336503 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":483}
I20260807 08:37:35.337245 31725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.328818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41456 (local address 127.9.174.2:41893)
0807 08:37:35.329480 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:35.329511 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:35.329542 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:37:35.329639 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:35.329644 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:35.329882 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:35.330076 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:35.330619 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.331868 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.335125 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.335801 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.335905 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.336089 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:35.336303 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 08:37:35.336442 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":168,"threads_started":1}
I20260807 08:37:35.337531 31724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.328554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48006 (local address 127.9.174.3:46317)
0807 08:37:35.329443 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:35.329459 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:35.329487 (+ 28us) server_negotiation.cc:407] Connection header received
0807 08:37:35.329590 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:35.329595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:35.330047 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:35.330662 (+ 615us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:35.330826 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.331732 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.333974 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.334966 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.336975 (+ 2009us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.337157 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:35.337207 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 08:37:35.337324 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":220,"threads_started":1}
I20260807 08:37:35.337579 31726 tablet_bootstrap.cc:492] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86: No bootstrap required, opened a new log
I20260807 08:37:35.337708 31726 ts_tablet_manager.cc:1397] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:37:35.338129 31726 raft_consensus.cc:348] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:35.338261 31726 raft_consensus.cc:374] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:35.338321 31726 raft_consensus.cc:729] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94499e8e1b714946bf6d45bcfeafdb86, State: Initialized, Role: FOLLOWER
I20260807 08:37:35.338503 31726 consensus_queue.cc:260] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:35.338902 31726 ts_tablet_manager.cc:1428] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:35.341331 31613 tablet_service.cc:1467] Processing CreateTablet for tablet e808904c387c4d088f4942c47d853052 (DEFAULT_TABLE table=test-table [id=0acf8c7d2b56497bae9ec33c9f1b3a3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:35.341325 31402 tablet_service.cc:1467] Processing CreateTablet for tablet e808904c387c4d088f4942c47d853052 (DEFAULT_TABLE table=test-table [id=0acf8c7d2b56497bae9ec33c9f1b3a3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:35.342640 31402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e808904c387c4d088f4942c47d853052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:35.342659 31613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e808904c387c4d088f4942c47d853052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:35.350526 31729 tablet_bootstrap.cc:492] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce: Bootstrap starting.
I20260807 08:37:35.351140 31730 tablet_bootstrap.cc:492] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a: Bootstrap starting.
I20260807 08:37:35.353736 31730 tablet_bootstrap.cc:654] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:35.353896 31729 tablet_bootstrap.cc:654] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:35.354691 31730 log.cc:824] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:35.355043 31729 log.cc:824] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:35.356927 31730 tablet_bootstrap.cc:492] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a: No bootstrap required, opened a new log
I20260807 08:37:35.356935 31729 tablet_bootstrap.cc:492] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce: No bootstrap required, opened a new log
I20260807 08:37:35.357159 31729 ts_tablet_manager.cc:1397] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 08:37:35.357159 31730 ts_tablet_manager.cc:1397] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 08:37:35.361063 31730 raft_consensus.cc:348] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:35.361061 31729 raft_consensus.cc:348] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:35.361340 31730 raft_consensus.cc:374] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:35.361361 31729 raft_consensus.cc:374] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:35.361399 31730 raft_consensus.cc:729] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83d9331d2ca54303a1f19c963d20017a, State: Initialized, Role: FOLLOWER
I20260807 08:37:35.361438 31729 raft_consensus.cc:729] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 333284d47de5482393fdc69fdc47e8ce, State: Initialized, Role: FOLLOWER
I20260807 08:37:35.361932 31730 consensus_queue.cc:260] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:35.361932 31729 consensus_queue.cc:260] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:35.362694 31512 heartbeater.cc:503] Master 127.9.174.62:45261 was elected leader, sending a full tablet report...
I20260807 08:37:35.363168 31729 ts_tablet_manager.cc:1428] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 08:37:35.363199 31711 heartbeater.cc:503] Master 127.9.174.62:45261 was elected leader, sending a full tablet report...
I20260807 08:37:35.363754 31730 ts_tablet_manager.cc:1428] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 08:37:35.420444 31734 raft_consensus.cc:482] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:35.420806 31734 raft_consensus.cc:504] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:35.422446 31734 leader_election.cc:288] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317), 94499e8e1b714946bf6d45bcfeafdb86 (127.9.174.1:42079)
I20260807 08:37:35.427748 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.422741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:46317 (local address 127.9.174.2:51701)
0807 08:37:35.423021 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:35.423043 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:35.423170 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:35.423517 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:35.423520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:35.423535 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:35.423717 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:35.423728 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.424802 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.424805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:35.426203 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:35.426216 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.427178 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.427181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:35.427330 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.427391 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 08:37:35.427477 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 08:37:35.427573 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":8}
I20260807 08:37:35.427755 31724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.422957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:51701 (local address 127.9.174.3:46317)
0807 08:37:35.423149 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:35.423156 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:35.423180 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:37:35.423293 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:35.423297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:35.423363 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:35.423463 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:35.423866 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.424682 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.426365 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.427048 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.427183 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.427273 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:35.427520 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 08:37:35.427606 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 08:37:35.427759 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.422926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:42381 (local address 127.9.174.1:42079)
0807 08:37:35.423347 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:35.423353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:35.423516 (+ 163us) server_negotiation.cc:407] Connection header received
0807 08:37:35.423647 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:35.423650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:35.423719 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:35.423863 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:35.424160 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.424900 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.426365 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.427147 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.427236 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.427427 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:35.427512 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 08:37:35.427603 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":184,"threads_started":1}
I20260807 08:37:35.428256 31736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.422779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:42079 (local address 127.9.174.2:42381)
0807 08:37:35.423391 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:35.423401 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:35.423529 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:35.423876 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:35.423879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:35.423903 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:35.424026 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:35.424032 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.425022 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.425025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:35.426207 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:35.426216 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.427259 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.427262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:35.427394 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.427411 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 08:37:35.427458 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:37:35.428105 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":60,"thread_start_us":204,"threads_started":1}
I20260807 08:37:35.428578 31641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e808904c387c4d088f4942c47d853052" candidate_uuid: "83d9331d2ca54303a1f19c963d20017a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "333284d47de5482393fdc69fdc47e8ce" is_pre_election: true
I20260807 08:37:35.429029 31641 raft_consensus.cc:2454] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83d9331d2ca54303a1f19c963d20017a in term 0.
I20260807 08:37:35.428982 31220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e808904c387c4d088f4942c47d853052" candidate_uuid: "83d9331d2ca54303a1f19c963d20017a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94499e8e1b714946bf6d45bcfeafdb86" is_pre_election: true
I20260807 08:37:35.429306 31220 raft_consensus.cc:2454] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83d9331d2ca54303a1f19c963d20017a in term 0.
I20260807 08:37:35.429705 31336 leader_election.cc:302] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 333284d47de5482393fdc69fdc47e8ce, 83d9331d2ca54303a1f19c963d20017a; no voters:
I20260807 08:37:35.430058 31734 raft_consensus.cc:2781] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:35.430140 31734 raft_consensus.cc:482] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:35.430171 31734 raft_consensus.cc:3037] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:35.431524 31734 raft_consensus.cc:504] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:35.431922 31734 leader_election.cc:288] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [CANDIDATE]: Term 1 election: Requested vote from peers 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317), 94499e8e1b714946bf6d45bcfeafdb86 (127.9.174.1:42079)
I20260807 08:37:35.432350 31641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e808904c387c4d088f4942c47d853052" candidate_uuid: "83d9331d2ca54303a1f19c963d20017a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "333284d47de5482393fdc69fdc47e8ce"
I20260807 08:37:35.432365 31220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e808904c387c4d088f4942c47d853052" candidate_uuid: "83d9331d2ca54303a1f19c963d20017a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94499e8e1b714946bf6d45bcfeafdb86"
I20260807 08:37:35.432483 31641 raft_consensus.cc:3037] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:35.432543 31220 raft_consensus.cc:3037] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:35.433790 31641 raft_consensus.cc:2454] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83d9331d2ca54303a1f19c963d20017a in term 1.
I20260807 08:37:35.433806 31220 raft_consensus.cc:2454] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83d9331d2ca54303a1f19c963d20017a in term 1.
I20260807 08:37:35.434105 31336 leader_election.cc:302] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 333284d47de5482393fdc69fdc47e8ce, 83d9331d2ca54303a1f19c963d20017a; no voters:
I20260807 08:37:35.434350 31734 raft_consensus.cc:2781] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:35.434616 31734 raft_consensus.cc:686] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [term 1 LEADER]: Becoming Leader. State: Replica: 83d9331d2ca54303a1f19c963d20017a, State: Running, Role: LEADER
I20260807 08:37:35.435057 31734 consensus_queue.cc:237] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:35.439373 31012 catalog_manager.cc:5495] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a reported cstate change: term changed from 0 to 1, leader changed from <none> to 83d9331d2ca54303a1f19c963d20017a (127.9.174.2). New cstate: current_term: 1 leader_uuid: "83d9331d2ca54303a1f19c963d20017a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } health_report { overall_health: UNKNOWN } } }
I20260807 08:37:35.463726 31725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.459675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41460 (local address 127.9.174.2:41893)
0807 08:37:35.459843 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:35.459850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:35.459882 (+ 32us) server_negotiation.cc:407] Connection header received
0807 08:37:35.459990 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:35.459992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:35.460063 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:35.460189 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:35.460477 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.461030 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.461912 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.462158 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.462275 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.462626 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:35.463245 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:35.463385 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:35.463428 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 08:37:35.463509 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 08:37:35.471506 31220 raft_consensus.cc:1261] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Refusing update from remote peer 83d9331d2ca54303a1f19c963d20017a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:35.471509 31641 raft_consensus.cc:1261] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce [term 1 FOLLOWER]: Refusing update from remote peer 83d9331d2ca54303a1f19c963d20017a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:35.472452 31740 consensus_queue.cc:1035] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [LEADER]: Connected to new peer: Peer: permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:35.472723 31734 consensus_queue.cc:1035] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [LEADER]: Connected to new peer: Peer: permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:35.479848 31750 mvcc.cc:205] Tried to move back new op lower bound from 7315832240001249280 to 7315832239865016320. Current Snapshot: MvccSnapshot[applied={T|T < 7315832240001249280}]
I20260807 08:37:35.480726 31748 mvcc.cc:205] Tried to move back new op lower bound from 7315832240001249280 to 7315832239865016320. Current Snapshot: MvccSnapshot[applied={T|T < 7315832240001249280}]
I20260807 08:37:35.491153 31011 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:41850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:35.491353 31011 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:41850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:35.494421 31011 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: Sending DeleteTablet for 3 replicas of tablet e808904c387c4d088f4942c47d853052
I20260807 08:37:35.495340 31402 tablet_service.cc:1514] Processing DeleteTablet for tablet e808904c387c4d088f4942c47d853052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:35 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:41456
I20260807 08:37:35.495462 31613 tablet_service.cc:1514] Processing DeleteTablet for tablet e808904c387c4d088f4942c47d853052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:35 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:48006
I20260807 08:37:35.495524 31181 tablet_service.cc:1514] Processing DeleteTablet for tablet e808904c387c4d088f4942c47d853052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:35 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:46432
I20260807 08:37:35.496137 31762 tablet_replica.cc:330] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a: stopping tablet replica
I20260807 08:37:35.496248 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 31067
I20260807 08:37:35.496565 31762 raft_consensus.cc:2229] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:37:35.501586 31762 raft_consensus.cc:2258] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:35.505048 31762 ts_tablet_manager.cc:1905] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:35.506621 31764 tablet_replica.cc:330] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce: stopping tablet replica
I20260807 08:37:35.507269 31764 raft_consensus.cc:2229] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:35.507911 31764 raft_consensus.cc:2258] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:35.508702 31762 ts_tablet_manager.cc:1918] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:35.508826 31762 log.cc:1196] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/e808904c387c4d088f4942c47d853052
I20260807 08:37:35.509223 31762 ts_tablet_manager.cc:1939] T e808904c387c4d088f4942c47d853052 P 83d9331d2ca54303a1f19c963d20017a: Deleting consensus metadata
I20260807 08:37:35.511045 30985 catalog_manager.cc:4841] TS 83d9331d2ca54303a1f19c963d20017a (127.9.174.2:41893): tablet e808904c387c4d088f4942c47d853052 (table test-table [id=0acf8c7d2b56497bae9ec33c9f1b3a3c]) successfully deleted
I20260807 08:37:35.512562 31764 ts_tablet_manager.cc:1905] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:35.515733 31764 ts_tablet_manager.cc:1918] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:35.515861 31764 log.cc:1196] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/e808904c387c4d088f4942c47d853052
I20260807 08:37:35.516254 31764 ts_tablet_manager.cc:1939] T e808904c387c4d088f4942c47d853052 P 333284d47de5482393fdc69fdc47e8ce: Deleting consensus metadata
W20260807 08:37:35.517019 30988 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:42079 (error 108)
I20260807 08:37:35.517514 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 31300
I20260807 08:37:35.520469 30987 catalog_manager.cc:4841] TS 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317): tablet e808904c387c4d088f4942c47d853052 (table test-table [id=0acf8c7d2b56497bae9ec33c9f1b3a3c]) successfully deleted
I20260807 08:37:35.525334 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.524850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:42079 (local address 127.0.0.1:46438)
0807 08:37:35.525057 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:35.525180 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:42079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 08:37:35.525628 30988 catalog_manager.cc:4584] TS 94499e8e1b714946bf6d45bcfeafdb86 (127.9.174.1:42079): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e808904c387c4d088f4942c47d853052: Network error: Client connection negotiation failed: client connection to 127.9.174.1:42079: connect: Connection refused (error 111)
I20260807 08:37:35.533568 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 31516
I20260807 08:37:35.549717 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 30955
I20260807 08:37:35.565456 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:45261
--webserver_interface=127.9.174.62
--webserver_port=38961
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:45261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:35.704764 31767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:35.705169 31767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:35.705217 31767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:35.709833 31767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:35.709949 31767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:35.709972 31767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:35.709991 31767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:35.710008 31767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:35.716414 31767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:45261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:45261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=38961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:35.717983 31767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:35.719370 31767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:35.730108 31779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:35.730108 31783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:35.730108 31777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:35.730609 31767 server_base.cc:1034] running on GCE node
I20260807 08:37:35.731371 31767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:35.732748 31767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:35.733953 31767 hybrid_clock.cc:648] HybridClock initialized: now 1786091855733921 us; error 47 us; skew 500 ppm
Aug 07 08:37:35 dist-test-slave-h3ft krb5kdc[30130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091855, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:35.737555 31767 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:37:35.738770 31767 webserver.cc:460] Webserver started at http://127.9.174.62:38961/ using document root <none> and password file <none>
I20260807 08:37:35.739549 31767 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:35.739624 31767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:35.744515 31767 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260807 08:37:35.747347 31793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:35.749030 31767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:37:35.749198 31767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "011c5d96f7874fe5b6bf66fe2a7adc11"
format_stamp: "Formatted at 2026-08-07 08:37:32 on dist-test-slave-h3ft"
server_key: "81b24a94c54ce3c4ace2b179c39d4f1e"
server_key_iv: "de7bd4b8e2a46e462af8728c63a7f38f"
server_key_version: "encryptionkey@0"
I20260807 08:37:35.749929 31767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:35.772195 31767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:35.776310 31767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:35.776571 31767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:35.785831 31767 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:45261
I20260807 08:37:35.786239 31866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:45261 every 8 connection(s)
I20260807 08:37:35.787402 31767 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:35.793049 31867 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:37:35.795114 31867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: Bootstrap starting.
I20260807 08:37:35.795610 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 31767
I20260807 08:37:35.796203 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:42079
--local_ip_for_outbound_sockets=127.9.174.1
--tserver_master_addrs=127.9.174.62:45261
--webserver_port=44597
--webserver_interface=127.9.174.1
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:37:35.818862 31867 log.cc:824] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:35.832324 31867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:37:35.832883 31867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11: Bootstrap complete.
I20260807 08:37:35.837788 31867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } }
I20260807 08:37:35.838291 31867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011c5d96f7874fe5b6bf66fe2a7adc11, State: Initialized, Role: FOLLOWER
I20260807 08:37:35.838865 31867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } }
I20260807 08:37:35.839042 31867 raft_consensus.cc:388] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:35.839123 31867 raft_consensus.cc:482] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:35.839251 31867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 2 FOLLOWER]: Advancing to term 3
I20260807 08:37:35.841578 31867 raft_consensus.cc:504] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } }
I20260807 08:37:35.841991 31867 leader_election.cc:302] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 011c5d96f7874fe5b6bf66fe2a7adc11; no voters:
I20260807 08:37:35.842448 31867 leader_election.cc:288] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 08:37:35.842562 31871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 08:37:35.842905 31871 raft_consensus.cc:686] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [term 3 LEADER]: Becoming Leader. State: Replica: 011c5d96f7874fe5b6bf66fe2a7adc11, State: Running, Role: LEADER
I20260807 08:37:35.843339 31871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } }
I20260807 08:37:35.844074 31867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:35.845022 31873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } } }
I20260807 08:37:35.845191 31873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:35.845985 31875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 011c5d96f7874fe5b6bf66fe2a7adc11. Latest consensus state: current_term: 3 leader_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011c5d96f7874fe5b6bf66fe2a7adc11" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45261 } } }
I20260807 08:37:35.846204 31875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 011c5d96f7874fe5b6bf66fe2a7adc11 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:35.846877 31883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:35.853130 31883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:35.853837 31883 catalog_manager.cc:1259] Loaded cluster ID: 7d4fb95049ee411183dfb9b75bea2c88
I20260807 08:37:35.853973 31883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 08:37:35.855235 31883 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f659adeb9ad google::LogMessage::Fail() at ??:0
@ 0x7f659adee88a google::LogMessage::SendToLog() at ??:0
@ 0x7f659adeb4de google::LogMessage::Flush() at ??:0
@ 0x7f659adeca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f65a572afdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f65a572bd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f65a5728579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f65a576c640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f659d03d3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f659d03dc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f659d03f3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f659d02e512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f659e6726db start_thread at ??:0
@ 0x7f65998e961f clone at ??:0
W20260807 08:37:35.968362 31869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:35.968727 31869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:35.968779 31869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:35.973223 31869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:35.973315 31869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:35.973419 31869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:37:35.980261 31869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:42079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=44597
--tserver_master_addrs=127.9.174.62:45261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:35.981873 31869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:35.983270 31869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:35.995442 31929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:35.995442 31931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:35.995772 31869 server_base.cc:1034] running on GCE node
W20260807 08:37:35.995829 31933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:35.996369 31869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:35.997256 31869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:35.998525 31869 hybrid_clock.cc:648] HybridClock initialized: now 1786091855998477 us; error 69 us; skew 500 ppm
Aug 07 08:37:35 dist-test-slave-h3ft krb5kdc[30130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091855, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:36.002103 31869 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:37:36.003417 31869 webserver.cc:460] Webserver started at http://127.9.174.1:44597/ using document root <none> and password file <none>
I20260807 08:37:36.004159 31869 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:36.004262 31869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:36.009454 31869 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:37:36.012431 31942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.014075 31869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:37:36.014221 31869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "94499e8e1b714946bf6d45bcfeafdb86"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h3ft"
server_key: "288fa9ba02a1ab11d040bca02c663fb2"
server_key_iv: "a8299ab9e4afec4df99a0c0aab22195c"
server_key_version: "encryptionkey@0"
I20260807 08:37:36.014955 31869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:36.035282 31869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:36.040400 31869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:36.040673 31869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:36.041505 31869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:36.042991 31952 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:37:36.050405 31869 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:37:36.050522 31869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.000s
I20260807 08:37:36.050638 31869 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:37:36.052949 31869 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:37:36.053032 31869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:36.053512 31952 tablet_bootstrap.cc:492] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86: Bootstrap starting.
I20260807 08:37:36.066290 31869 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:42079
I20260807 08:37:36.066574 32108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:42079 every 8 connection(s)
I20260807 08:37:36.067759 31869 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:36.077654 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 31869
I20260807 08:37:36.078311 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:41893
--local_ip_for_outbound_sockets=127.9.174.2
--tserver_master_addrs=127.9.174.62:45261
--webserver_port=44541
--webserver_interface=127.9.174.2
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:37:36.088371 31952 log.cc:824] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:36.098302 31952 tablet_bootstrap.cc:492] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:37:36.098799 31952 tablet_bootstrap.cc:492] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86: Bootstrap complete.
I20260807 08:37:36.100350 31952 ts_tablet_manager.cc:1397] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86: Time spent bootstrapping tablet: real 0.047s user 0.026s sys 0.018s
I20260807 08:37:36.103446 31952 raft_consensus.cc:348] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:36.103998 31952 raft_consensus.cc:729] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94499e8e1b714946bf6d45bcfeafdb86, State: Initialized, Role: FOLLOWER
I20260807 08:37:36.104615 31952 consensus_queue.cc:260] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:36.105861 31952 ts_tablet_manager.cc:1428] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
W20260807 08:37:36.167296 32114 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:36.070418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:49875)
0807 08:37:36.071075 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.071144 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:36.072186 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:36.166223 (+ 94037us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":177,"threads_started":1}
W20260807 08:37:36.167904 32109 heartbeater.cc:644] Failed to heartbeat to 127.9.174.62:45261 (0 consecutive failures): Network error: Failed to ping master at 127.9.174.62:45261: Client connection negotiation failed: client connection to 127.9.174.62:45261: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:37:36.168823 32114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.168516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:47089)
0807 08:37:36.168667 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.168760 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 08:37:36.169505 32114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.169281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:36043)
0807 08:37:36.169397 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.169446 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 08:37:36.169674 32109 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:37:36.240937 32115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:36.241345 32115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:36.241428 32115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:36.245713 32115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:36.245843 32115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:36.245945 32115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:37:36.252077 32115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:41893
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=44541
--tserver_master_addrs=127.9.174.62:45261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:36.253592 32115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:36.254935 32115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:36.267051 32129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:36.267151 32127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:36.267314 32115 server_base.cc:1034] running on GCE node
W20260807 08:37:36.267373 32133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:36.267838 32115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:36.268606 32115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:36.269794 32115 hybrid_clock.cc:648] HybridClock initialized: now 1786091856269757 us; error 54 us; skew 500 ppm
Aug 07 08:37:36 dist-test-slave-h3ft krb5kdc[30130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091856, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:36.273075 32115 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:37:36.274135 32115 webserver.cc:460] Webserver started at http://127.9.174.2:44541/ using document root <none> and password file <none>
I20260807 08:37:36.274777 32115 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:36.274857 32115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:36.279440 32115 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:37:36.281836 32143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.283345 32115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:36.283476 32115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "83d9331d2ca54303a1f19c963d20017a"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h3ft"
server_key: "36c65ec9cc60e4d23b538bb9ac9ea972"
server_key_iv: "56f38d67babe411cb206a94cbdf76fd9"
server_key_version: "encryptionkey@0"
I20260807 08:37:36.284123 32115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:36.311504 32115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:36.315685 32115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:36.315930 32115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:36.316727 32115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:36.317762 32115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:36.317821 32115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.317907 32115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:36.317948 32115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.329743 32115 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:41893
I20260807 08:37:36.329766 32334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:41893 every 8 connection(s)
I20260807 08:37:36.331084 32115 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:36.331831 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 32115
I20260807 08:37:36.332338 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:46317
--local_ip_for_outbound_sockets=127.9.174.3
--tserver_master_addrs=127.9.174.62:45261
--webserver_port=36183
--webserver_interface=127.9.174.3
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:37:36.337458 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.334331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:38737)
0807 08:37:36.335296 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.335514 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":334,"threads_started":1}
W20260807 08:37:36.338083 32335 heartbeater.cc:644] Failed to heartbeat to 127.9.174.62:45261 (0 consecutive failures): Network error: Failed to ping master at 127.9.174.62:45261: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
I20260807 08:37:36.338954 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.338621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:40809)
0807 08:37:36.338784 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.338859 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 08:37:36.339767 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.339493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:55467)
0807 08:37:36.339623 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.339687 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 08:37:36.339970 32335 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:37:36.467672 32338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:36.467988 32338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:36.468035 32338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:36.472150 32338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:36.472211 32338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:36.472268 32338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:37:36.477761 32338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:46317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=36183
--tserver_master_addrs=127.9.174.62:45261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:36.479121 32338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:36.480347 32338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:36.491833 32346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:36.491863 32349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:36.491838 32352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:36.492197 32338 server_base.cc:1034] running on GCE node
I20260807 08:37:36.492795 32338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:36.493642 32338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:36.494824 32338 hybrid_clock.cc:648] HybridClock initialized: now 1786091856494801 us; error 34 us; skew 500 ppm
Aug 07 08:37:36 dist-test-slave-h3ft krb5kdc[30130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091856, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:36.498039 32338 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:37:36.499179 32338 webserver.cc:460] Webserver started at http://127.9.174.3:36183/ using document root <none> and password file <none>
I20260807 08:37:36.499882 32338 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:36.499982 32338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:36.504637 32338 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 08:37:36.507315 32363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.508822 32338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 08:37:36.508971 32338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "333284d47de5482393fdc69fdc47e8ce"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h3ft"
server_key: "8e14f7588dfdd1aed94a7ad1dcc7b04c"
server_key_iv: "3aeeb6c9d3a25177db00abe56c6e5821"
server_key_version: "encryptionkey@0"
I20260807 08:37:36.509634 32338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:36.549458 32338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:36.553506 32338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:36.553771 32338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:36.554596 32338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:36.555805 32338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:36.555886 32338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.555981 32338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:36.556034 32338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.567586 32338 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:46317
I20260807 08:37:36.567613 32540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:46317 every 8 connection(s)
I20260807 08:37:36.568861 32338 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:37:36.573671 32546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.571473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:35563)
0807 08:37:36.572054 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.572188 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":145,"threads_started":1}
I20260807 08:37:36.574164 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 32338
W20260807 08:37:36.574208 32544 heartbeater.cc:644] Failed to heartbeat to 127.9.174.62:45261 (0 consecutive failures): Network error: Failed to ping master at 127.9.174.62:45261: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
I20260807 08:37:36.575079 32546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.574710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:39267)
0807 08:37:36.574850 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.574915 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 08:37:36.575857 32546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.575646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:37639)
0807 08:37:36.575750 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.575796 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 08:37:36.576066 32544 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 08:37:37.171168 32548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:37.170455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:59939)
0807 08:37:37.170859 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:37.170930 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":106,"threads_started":1}
I20260807 08:37:37.341321 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:37.340692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:47683)
0807 08:37:37.341096 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:37.341166 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":122,"threads_started":1}
I20260807 08:37:37.577260 32550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:37.576730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:58431)
0807 08:37:37.577053 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:37.577126 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":103,"threads_started":1}
I20260807 08:37:37.917246 32551 raft_consensus.cc:482] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:37.917524 32551 raft_consensus.cc:504] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:37.919034 32551 leader_election.cc:288] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317), 83d9331d2ca54303a1f19c963d20017a (127.9.174.2:41893)
Aug 07 08:37:37 dist-test-slave-h3ft krb5kdc[30130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091855, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.3@KRBTEST.COM
Aug 07 08:37:37 dist-test-slave-h3ft krb5kdc[30130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091855, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.2@KRBTEST.COM
I20260807 08:37:37.936048 32559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:37.919604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:35639 (local address 127.9.174.3:46317)
0807 08:37:37.920187 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:37.920212 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:37.920280 (+ 68us) server_negotiation.cc:407] Connection header received
0807 08:37:37.921300 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:37.921309 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:37.921683 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:37.922040 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:37.923611 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.924669 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.926042 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.926564 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.926752 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:37.930042 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:37.930048 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:37.930063 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:37.930101 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:37.932799 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:37.933580 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:37.933590 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:37.933595 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:37.933736 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:37.934157 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:37.934162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:37.934164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:37.934670 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:37.934827 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:37.935496 (+ 669us) server_negotiation.cc:299] Negotiation successful
0807 08:37:37.935794 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":171,"threads_started":1}
I20260807 08:37:37.936048 32558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:37.919305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:41893 (local address 127.9.174.1:56117)
0807 08:37:37.919817 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:37.919835 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:37.919932 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:37.922095 (+ 2163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:37.922103 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:37.922746 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:37.923422 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:37.923440 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.924824 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.924828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:37.925884 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:37.925894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.926820 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.926824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:37.926923 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:37.927798 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:37.927812 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:37.929907 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:37.933178 (+ 3271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:37.933181 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:37.933183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:37.933454 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:37.933836 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:37.933839 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:37.933841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:37.934036 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:37.934795 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:37.934802 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:37.935145 (+ 343us) client_negotiation.cc:769] Sending connection context
0807 08:37:37.935365 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 08:37:37.935805 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":299,"spinlock_wait_cycles":30976,"thread_start_us":85,"threads_started":1}
I20260807 08:37:37.936049 32560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:37.919617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:56117 (local address 127.9.174.2:41893)
0807 08:37:37.920193 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:37.920212 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:37.920286 (+ 74us) server_negotiation.cc:407] Connection header received
0807 08:37:37.921236 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:37.921245 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:37.921682 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:37.922039 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:37.923614 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.924671 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.926037 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.926598 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.926888 (+ 290us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:37.930127 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:37.930133 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:37.930144 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:37.930176 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:37.933052 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:37.933583 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:37.933590 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:37.933595 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:37.933735 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:37.934135 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:37.934139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:37.934141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:37.934669 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:37.934826 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:37.935488 (+ 662us) server_negotiation.cc:299] Negotiation successful
0807 08:37:37.935794 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":150,"threads_started":1}
I20260807 08:37:37.936092 32553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:37.919303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:46317 (local address 127.9.174.1:35639)
0807 08:37:37.919723 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:37.919741 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:37.919911 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:37.922095 (+ 2184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:37.922103 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:37.922747 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:37.923422 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:37.923440 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.924841 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.924843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:37.925883 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:37.925893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.926705 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.926709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:37.926878 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:37.927744 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:37.927762 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:37.929763 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:37.933001 (+ 3238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:37.933008 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:37.933023 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:37.933431 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:37.933838 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:37.933843 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:37.933846 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:37.934059 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:37.934795 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:37.934802 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:37.935145 (+ 343us) client_negotiation.cc:769] Sending connection context
0807 08:37:37.935365 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 08:37:37.935895 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":388,"spinlock_wait_cycles":6528,"thread_start_us":117,"threads_started":1}
W20260807 08:37:37.939963 31944 leader_election.cc:341] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83d9331d2ca54303a1f19c963d20017a (127.9.174.2:41893): Not found: Tablet not found: e808904c387c4d088f4942c47d853052
W20260807 08:37:37.940176 31945 leader_election.cc:341] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317): Not found: Tablet not found: e808904c387c4d088f4942c47d853052
I20260807 08:37:37.940295 31945 leader_election.cc:302] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94499e8e1b714946bf6d45bcfeafdb86; no voters: 333284d47de5482393fdc69fdc47e8ce, 83d9331d2ca54303a1f19c963d20017a
I20260807 08:37:37.940630 32551 raft_consensus.cc:2726] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:37:38.172606 32553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:38.172194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:38577)
0807 08:37:38.172382 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:38.172465 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 08:37:38.342674 32562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:38.342154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:48491)
0807 08:37:38.342452 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:38.342515 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":96,"threads_started":1}
I20260807 08:37:38.578837 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:38.578280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:50971)
0807 08:37:38.578629 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:38.578704 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":106,"threads_started":1}
I20260807 08:37:39.174494 32566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:39.173732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:50229)
0807 08:37:39.174185 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:39.174278 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":148,"threads_started":1}
I20260807 08:37:39.344321 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:39.343698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:42417)
0807 08:37:39.344068 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:39.344147 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":116,"threads_started":1}
I20260807 08:37:39.580528 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:39.579917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:56335)
0807 08:37:39.580286 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:39.580365 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":121,"threads_started":1}
I20260807 08:37:39.671717 32570 raft_consensus.cc:482] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:39.671957 32570 raft_consensus.cc:504] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:39.672504 32570 leader_election.cc:288] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317), 83d9331d2ca54303a1f19c963d20017a (127.9.174.2:41893)
W20260807 08:37:39.673871 31945 leader_election.cc:341] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317): Not found: Tablet not found: e808904c387c4d088f4942c47d853052
W20260807 08:37:39.674050 31944 leader_election.cc:341] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83d9331d2ca54303a1f19c963d20017a (127.9.174.2:41893): Not found: Tablet not found: e808904c387c4d088f4942c47d853052
I20260807 08:37:39.674153 31944 leader_election.cc:302] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94499e8e1b714946bf6d45bcfeafdb86; no voters: 333284d47de5482393fdc69fdc47e8ce, 83d9331d2ca54303a1f19c963d20017a
I20260807 08:37:39.674346 32570 raft_consensus.cc:2726] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:37:40.176115 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:40.175547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:47363)
0807 08:37:40.175860 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:40.175942 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":97,"threads_started":1}
I20260807 08:37:40.346105 32574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:40.345493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:58929)
0807 08:37:40.345839 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:40.345906 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":128,"threads_started":1}
I20260807 08:37:40.581996 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:40.581467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:44451)
0807 08:37:40.581798 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:40.581866 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":102,"threads_started":1}
I20260807 08:37:41.177922 32577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:41.177334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:49979)
0807 08:37:41.177713 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:41.177780 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":107,"threads_started":1}
I20260807 08:37:41.347898 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:41.347323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:48285)
0807 08:37:41.347682 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:41.347759 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":112,"threads_started":1}
I20260807 08:37:41.583678 32580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:41.583076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:39107)
0807 08:37:41.583423 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:41.583492 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":101,"threads_started":1}
I20260807 08:37:41.933701 32582 raft_consensus.cc:482] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:41.933938 32582 raft_consensus.cc:504] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:41.934574 32582 leader_election.cc:288] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317), 83d9331d2ca54303a1f19c963d20017a (127.9.174.2:41893)
W20260807 08:37:41.935781 31944 leader_election.cc:341] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83d9331d2ca54303a1f19c963d20017a (127.9.174.2:41893): Not found: Tablet not found: e808904c387c4d088f4942c47d853052
W20260807 08:37:41.935926 31945 leader_election.cc:341] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317): Not found: Tablet not found: e808904c387c4d088f4942c47d853052
I20260807 08:37:41.936046 31945 leader_election.cc:302] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94499e8e1b714946bf6d45bcfeafdb86; no voters: 333284d47de5482393fdc69fdc47e8ce, 83d9331d2ca54303a1f19c963d20017a
I20260807 08:37:41.936242 32582 raft_consensus.cc:2726] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:37:42.179528 32585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:42.178899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:44409)
0807 08:37:42.179310 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:42.179389 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":165,"threads_started":1}
I20260807 08:37:42.349632 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:42.349020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:56395)
0807 08:37:42.349406 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:42.349483 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":151,"threads_started":1}
I20260807 08:37:42.585371 32588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:42.584712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:51257)
0807 08:37:42.585160 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:42.585234 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":153,"threads_started":1}
I20260807 08:37:43.181267 32589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.180625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:59031)
0807 08:37:43.181005 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:43.181102 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":114,"threads_started":1}
I20260807 08:37:43.351287 32591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.350608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:58893)
0807 08:37:43.351059 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:43.351144 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":118,"threads_started":1}
I20260807 08:37:43.587134 32592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.586405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:45803)
0807 08:37:43.586819 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:43.586893 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":113,"threads_started":1}
I20260807 08:37:43.638620 32594 raft_consensus.cc:482] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:43.638814 32594 raft_consensus.cc:504] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:43.639618 32594 leader_election.cc:288] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317), 83d9331d2ca54303a1f19c963d20017a (127.9.174.2:41893)
W20260807 08:37:43.640597 31944 leader_election.cc:341] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83d9331d2ca54303a1f19c963d20017a (127.9.174.2:41893): Not found: Tablet not found: e808904c387c4d088f4942c47d853052
W20260807 08:37:43.640774 31945 leader_election.cc:341] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317): Not found: Tablet not found: e808904c387c4d088f4942c47d853052
I20260807 08:37:43.640866 31945 leader_election.cc:302] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94499e8e1b714946bf6d45bcfeafdb86; no voters: 333284d47de5482393fdc69fdc47e8ce, 83d9331d2ca54303a1f19c963d20017a
I20260807 08:37:43.641036 32594 raft_consensus.cc:2726] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:37:44.183140 32595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.182427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:54343)
0807 08:37:44.182807 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.182883 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":109,"threads_started":1}
I20260807 08:37:44.352963 32599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.352319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:39909)
0807 08:37:44.352703 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.352782 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":115,"threads_started":1}
I20260807 08:37:44.588620 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.588112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:46351)
0807 08:37:44.588409 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.588470 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":100,"threads_started":1}
I20260807 08:37:45.184798 32604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:45.184207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:38859)
0807 08:37:45.184589 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:45.184662 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":119,"threads_started":1}
I20260807 08:37:45.354667 32607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:45.354040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:40151)
0807 08:37:45.354429 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:45.354523 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":107,"threads_started":1}
I20260807 08:37:45.590279 32609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:45.589650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:46345)
0807 08:37:45.590050 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:45.590118 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":123,"threads_started":1}
I20260807 08:37:46.186456 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.185906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:43043)
0807 08:37:46.186238 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.186305 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":108,"threads_started":1}
I20260807 08:37:46.356292 32611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.355689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:43017)
0807 08:37:46.356064 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.356145 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":96,"threads_started":1}
I20260807 08:37:46.592073 32612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.591348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:43141)
0807 08:37:46.591852 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.591933 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":138,"threads_started":1}
I20260807 08:37:47.188184 32614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:47.187560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:41699)
0807 08:37:47.187955 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:47.188041 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":130,"threads_started":1}
I20260807 08:37:47.357806 32616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:47.357284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:48059)
0807 08:37:47.357595 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:47.357662 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":95,"threads_started":1}
I20260807 08:37:47.593889 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:47.593280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:56945)
0807 08:37:47.593656 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:47.593738 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":114,"threads_started":1}
I20260807 08:37:48.028863 32619 raft_consensus.cc:482] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:48.029070 32619 raft_consensus.cc:504] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "333284d47de5482393fdc69fdc47e8ce" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46317 } } peers { permanent_uuid: "83d9331d2ca54303a1f19c963d20017a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 41893 } } peers { permanent_uuid: "94499e8e1b714946bf6d45bcfeafdb86" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 42079 } }
I20260807 08:37:48.029642 32619 leader_election.cc:288] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317), 83d9331d2ca54303a1f19c963d20017a (127.9.174.2:41893)
W20260807 08:37:48.030802 31944 leader_election.cc:341] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83d9331d2ca54303a1f19c963d20017a (127.9.174.2:41893): Not found: Tablet not found: e808904c387c4d088f4942c47d853052
W20260807 08:37:48.031028 31945 leader_election.cc:341] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 333284d47de5482393fdc69fdc47e8ce (127.9.174.3:46317): Not found: Tablet not found: e808904c387c4d088f4942c47d853052
I20260807 08:37:48.031136 31945 leader_election.cc:302] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94499e8e1b714946bf6d45bcfeafdb86; no voters: 333284d47de5482393fdc69fdc47e8ce, 83d9331d2ca54303a1f19c963d20017a
I20260807 08:37:48.031334 32619 raft_consensus.cc:2726] T e808904c387c4d088f4942c47d853052 P 94499e8e1b714946bf6d45bcfeafdb86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:37:48.189921 32620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.189348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:35729)
0807 08:37:48.189697 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:48.189775 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":108,"threads_started":1}
I20260807 08:37:48.359905 32621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.359113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:34707)
0807 08:37:48.359608 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:48.359698 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":189,"threads_started":1}
I20260807 08:37:48.595647 32622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.595064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:50003)
0807 08:37:48.595441 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:48.595513 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":126,"threads_started":1}
I20260807 08:37:49.191686 32624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.191063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:53909)
0807 08:37:49.191478 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:49.191547 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":132,"threads_started":1}
I20260807 08:37:49.361842 32625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.361089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:36069)
0807 08:37:49.361565 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:49.361656 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":163,"threads_started":1}
I20260807 08:37:49.597414 32627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.596773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:58579)
0807 08:37:49.597179 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:49.597256 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":131,"threads_started":1}
I20260807 08:37:50.193517 32629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:50.192910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:33315)
0807 08:37:50.193292 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:50.193374 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":123,"threads_started":1}
I20260807 08:37:50.363453 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:50.362882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:54789)
0807 08:37:50.363242 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:50.363315 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":135,"threads_started":1}
I20260807 08:37:50.598982 32631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:50.598433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:40517)
0807 08:37:50.598764 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:50.598830 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":108,"threads_started":1}
I20260807 08:37:51.195467 32632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:51.194851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.1:46819)
0807 08:37:51.195247 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:51.195325 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":98,"threads_started":1}
I20260807 08:37:51.365187 32633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:51.364611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.2:33033)
0807 08:37:51.364966 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:51.365047 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":110,"threads_started":1}
I20260807 08:37:51.576761 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 31869
I20260807 08:37:51.593923 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 32115
I20260807 08:37:51.602294 32635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:51.600922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:45261 (local address 127.9.174.3:39293)
0807 08:37:51.602012 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:51.602111 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":949,"thread_start_us":805,"threads_started":1}
I20260807 08:37:51.609609 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 32338
2026-08-07T08:37:51Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (20257 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:51 dist-test-slave-h3ft krb5kdc[32638](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:51 dist-test-slave-h3ft krb5kdc[32638](info): set up 2 sockets
Aug 07 08:37:51 dist-test-slave-h3ft krb5kdc[32638](info): commencing operation
krb5kdc: starting...
W20260807 08:37:52.892304 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:37:52 dist-test-slave-h3ft krb5kdc[32638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:37:53.034029 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:43599
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:43599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:53.167148 32664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:53.167477 32664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:53.167529 32664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:53.171553 32664 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 08:37:53.171602 32664 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 08:37:53.171617 32664 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 08:37:53.171631 32664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:53.171649 32664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:53.171666 32664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:53.171682 32664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:53.171697 32664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:53.177134 32664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:43599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:43599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:53.178380 32664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:53.179597 32664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:53.190507 32670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:53.190615 32664 server_base.cc:1034] running on GCE node
W20260807 08:37:53.190543 32671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:53.190456 32674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:53.191675 32664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:53.193174 32664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:53.194389 32664 hybrid_clock.cc:648] HybridClock initialized: now 1786091873194372 us; error 42 us; skew 500 ppm
Aug 07 08:37:53 dist-test-slave-h3ft krb5kdc[32638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091873, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:53.197882 32664 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:37:53.199056 32664 webserver.cc:460] Webserver started at http://127.9.174.62:44787/ using document root <none> and password file <none>
I20260807 08:37:53.199738 32664 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:53.199833 32664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:53.200112 32664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:53.202711 32664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "bbeaeef13c814174b459f3f7eedd66b2"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h3ft"
server_key: "fa11fb4127875411ce1036c3dbd720d0"
server_key_iv: "e263d3838e4a75220cebdafbca95a662"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.203466 32664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "bbeaeef13c814174b459f3f7eedd66b2"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h3ft"
server_key: "fa11fb4127875411ce1036c3dbd720d0"
server_key_iv: "e263d3838e4a75220cebdafbca95a662"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.207809 32664 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:37:53.210638 32682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.212195 32664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:53.212347 32664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "bbeaeef13c814174b459f3f7eedd66b2"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h3ft"
server_key: "fa11fb4127875411ce1036c3dbd720d0"
server_key_iv: "e263d3838e4a75220cebdafbca95a662"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.212467 32664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:53.225833 32664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:53.229715 32664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:53.229945 32664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:53.238314 32664 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:43599
I20260807 08:37:53.238667 303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:43599 every 8 connection(s)
I20260807 08:37:53.239526 32664 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:53.242141 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 32664
I20260807 08:37:53.242257 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:37:53.242658 9912 external_mini_cluster.cc:1428] Setting key d03bd16b0dad7e3be43a1ce9f1fd0afa
I20260807 08:37:53.243230 304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:53.251935 304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Bootstrap starting.
Aug 07 08:37:53 dist-test-slave-h3ft krb5kdc[32638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:53.255105 304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:53.256261 304 log.cc:824] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:53.258683 304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: No bootstrap required, opened a new log
I20260807 08:37:53.259897 308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:53.244476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38804 (local address 127.9.174.62:43599)
0807 08:37:53.245016 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:53.245039 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:53.245090 (+ 51us) server_negotiation.cc:407] Connection header received
0807 08:37:53.246022 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:53.246049 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:53.246535 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:53.246962 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:53.247778 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.249401 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.250358 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.251165 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.251331 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:53.254205 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:53.254212 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:53.254230 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:53.254267 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:53.256481 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:53.256924 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:53.256931 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:53.256936 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:53.257124 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:53.257559 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:53.257562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:53.257564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:53.258047 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:53.258277 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:53.258588 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 08:37:53.258979 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":151,"threads_started":1}
I20260807 08:37:53.262904 304 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } }
I20260807 08:37:53.263247 304 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:53.263334 304 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbeaeef13c814174b459f3f7eedd66b2, State: Initialized, Role: FOLLOWER
I20260807 08:37:53.263891 304 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } }
I20260807 08:37:53.264078 304 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:53.264171 304 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:53.264282 304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:53.265324 304 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } }
I20260807 08:37:53.265652 304 leader_election.cc:302] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbeaeef13c814174b459f3f7eedd66b2; no voters:
I20260807 08:37:53.265941 304 leader_election.cc:288] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:53.266166 311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:53.266440 311 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 1 LEADER]: Becoming Leader. State: Replica: bbeaeef13c814174b459f3f7eedd66b2, State: Running, Role: LEADER
I20260807 08:37:53.266803 311 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } }
I20260807 08:37:53.267457 304 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:53.268734 316 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbeaeef13c814174b459f3f7eedd66b2. Latest consensus state: current_term: 1 leader_uuid: "bbeaeef13c814174b459f3f7eedd66b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } } }
I20260807 08:37:53.268883 316 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:53.269277 312 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbeaeef13c814174b459f3f7eedd66b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } } }
I20260807 08:37:53.269342 325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:53.269429 312 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:53.273900 325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:53.280381 325 catalog_manager.cc:1347] Generated new cluster ID: 2a5bb6cefc8141aa9b5f8e6a47fdb26b
I20260807 08:37:53.280474 325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:53.295145 325 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:53.296499 325 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:53.307225 325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Generated new TSK 0
I20260807 08:37:53.308199 325 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:37:53.353746 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43599
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:53.495278 340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:53.495612 340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:53.495663 340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:53.499787 340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:53.499854 340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:53.499914 340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:37:53.505590 340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:53.506946 340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:53.508285 340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:53.520176 353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:53.520187 345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:53.520187 347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:53.520617 340 server_base.cc:1034] running on GCE node
I20260807 08:37:53.521055 340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:53.521799 340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:53.523082 340 hybrid_clock.cc:648] HybridClock initialized: now 1786091873523050 us; error 46 us; skew 500 ppm
Aug 07 08:37:53 dist-test-slave-h3ft krb5kdc[32638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091873, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:53.526391 340 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:37:53.527508 340 webserver.cc:460] Webserver started at http://127.9.174.1:40153/ using document root <none> and password file <none>
I20260807 08:37:53.528216 340 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:53.528319 340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:53.528601 340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:53.531279 340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "757fe0884d894fa380b34c2928462b9f"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h3ft"
server_key: "43fdd10fa3b1658ba39cf0150d663ca7"
server_key_iv: "7d542f8083822e3778e5dc39b0eae33d"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.531970 340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "757fe0884d894fa380b34c2928462b9f"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h3ft"
server_key: "43fdd10fa3b1658ba39cf0150d663ca7"
server_key_iv: "7d542f8083822e3778e5dc39b0eae33d"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.536718 340 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:37:53.539661 364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.541215 340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:37:53.541366 340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "757fe0884d894fa380b34c2928462b9f"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h3ft"
server_key: "43fdd10fa3b1658ba39cf0150d663ca7"
server_key_iv: "7d542f8083822e3778e5dc39b0eae33d"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.541494 340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:53.556231 340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:53.559825 340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:53.560066 340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:53.560791 340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:53.561806 340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:53.561861 340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.561909 340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:53.561959 340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.572852 340 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:46005
I20260807 08:37:53.572875 529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:46005 every 8 connection(s)
I20260807 08:37:53.573969 340 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:53.584460 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 340
I20260807 08:37:53.584661 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:37:53.585089 9912 external_mini_cluster.cc:1428] Setting key 69d7fb25899b4fa189b6da3f274c168d
Aug 07 08:37:53 dist-test-slave-h3ft krb5kdc[32638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091873, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:53.591415 308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:53.576257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:37291 (local address 127.9.174.62:43599)
0807 08:37:53.576438 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:53.576442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:53.577459 (+ 1017us) server_negotiation.cc:407] Connection header received
0807 08:37:53.579162 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:53.579166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:53.579257 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:53.579365 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:53.580932 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.581547 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.582888 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.583222 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.583351 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:53.586835 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:53.586839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:53.586841 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:53.586871 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:53.588765 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:53.589376 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:53.589382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:53.589387 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:53.589499 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:53.589868 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:53.589874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:53.589878 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:53.590206 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:53.590330 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:53.591026 (+ 696us) server_negotiation.cc:299] Negotiation successful
0807 08:37:53.591203 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 08:37:53.592540 533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:53.576722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:37291)
0807 08:37:53.577272 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:53.577323 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:53.578833 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:53.579596 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:53.579604 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:53.580082 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:53.580727 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:53.580744 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.581706 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.581710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:53.582728 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:53.582739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.583420 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.583424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:53.583659 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:53.584537 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:53.584556 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:53.586633 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:53.588907 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:53.588913 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:53.588929 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:53.589260 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:53.589603 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:53.589605 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:53.589607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:53.589771 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:53.590333 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:53.590344 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:53.590688 (+ 344us) client_negotiation.cc:769] Sending connection context
0807 08:37:53.590927 (+ 239us) client_negotiation.cc:240] Negotiation successful
0807 08:37:53.591409 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":147,"threads_started":1}
I20260807 08:37:53.594231 530 heartbeater.cc:347] Connected to a master server at 127.9.174.62:43599
I20260807 08:37:53.594588 530 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:53.595479 530 heartbeater.cc:511] Master 127.9.174.62:43599 requested a full tablet report, sending...
I20260807 08:37:53.597869 32714 ts_manager.cc:194] Registered new tserver with Master: 757fe0884d894fa380b34c2928462b9f (127.9.174.1:46005)
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
I20260807 08:37:53.599850 32714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:37291
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:37:53.622892 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43599
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:53.764441 540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:53.764778 540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:53.764828 540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:53.769168 540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:53.769239 540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:53.769304 540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:37:53.775098 540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:53.776377 540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:53.777575 540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:53.789443 547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:53.789481 549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:53.789513 551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:53.789840 540 server_base.cc:1034] running on GCE node
I20260807 08:37:53.790429 540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:53.791334 540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:53.792532 540 hybrid_clock.cc:648] HybridClock initialized: now 1786091873792494 us; error 51 us; skew 500 ppm
Aug 07 08:37:53 dist-test-slave-h3ft krb5kdc[32638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091873, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:53.795823 540 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:37:53.796934 540 webserver.cc:460] Webserver started at http://127.9.174.2:42187/ using document root <none> and password file <none>
I20260807 08:37:53.797648 540 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:53.797750 540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:53.798031 540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:53.800650 540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "47fa071b785242bdb0861aa94d868819"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h3ft"
server_key: "029cb0f1a7c134157aef0aa0919bddc3"
server_key_iv: "6b84a0d3876d478ed778d5b4bf95bff0"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.801326 540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "47fa071b785242bdb0861aa94d868819"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h3ft"
server_key: "029cb0f1a7c134157aef0aa0919bddc3"
server_key_iv: "6b84a0d3876d478ed778d5b4bf95bff0"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.805846 540 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 08:37:53.808733 559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.810369 540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:37:53.810555 540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "47fa071b785242bdb0861aa94d868819"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h3ft"
server_key: "029cb0f1a7c134157aef0aa0919bddc3"
server_key_iv: "6b84a0d3876d478ed778d5b4bf95bff0"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.810704 540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:53.837531 540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:53.841377 540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:53.841652 540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:53.842545 540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:53.843777 540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:53.843858 540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.843945 540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:53.844000 540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.855290 540 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:37327
I20260807 08:37:53.855314 742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:37327 every 8 connection(s)
I20260807 08:37:53.856448 540 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:53.863323 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 540
I20260807 08:37:53.863479 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:37:53.863894 9912 external_mini_cluster.cc:1428] Setting key 28b69adb8deb1e3f50c5208abbb1f7e9
Aug 07 08:37:53 dist-test-slave-h3ft krb5kdc[32638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091873, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:53.875174 308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:53.858758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:39815 (local address 127.9.174.62:43599)
0807 08:37:53.858945 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:53.858949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:53.860083 (+ 1134us) server_negotiation.cc:407] Connection header received
0807 08:37:53.862044 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:53.862049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:53.862134 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:53.862270 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:53.864167 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.864787 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.866639 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.866883 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.866976 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:53.870632 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:53.870637 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:53.870639 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:53.870666 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:53.872260 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:53.873039 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:53.873042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:53.873043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:53.873134 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:53.873596 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:53.873599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:53.873600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:53.873820 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:53.873901 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:53.874634 (+ 733us) server_negotiation.cc:299] Negotiation successful
0807 08:37:53.874811 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 08:37:53.876046 748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:53.859240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:39815)
0807 08:37:53.859864 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:53.859920 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:53.861754 (+ 1834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:53.862493 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:53.862501 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:53.863066 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:53.863926 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:53.863954 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.864977 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.864981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:53.866485 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:53.866502 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.867096 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.867104 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:53.867406 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:53.868358 (+ 952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:53.868389 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:53.870426 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:53.872389 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:53.872398 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:53.872420 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:53.872912 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:53.873236 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:53.873242 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:53.873245 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:53.873475 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:53.873940 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:53.873952 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:53.874317 (+ 365us) client_negotiation.cc:769] Sending connection context
0807 08:37:53.874504 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 08:37:53.874831 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":149,"threads_started":1}
I20260807 08:37:53.877496 744 heartbeater.cc:347] Connected to a master server at 127.9.174.62:43599
I20260807 08:37:53.877792 744 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
I20260807 08:37:53.878472 744 heartbeater.cc:511] Master 127.9.174.62:43599 requested a full tablet report, sending...
I20260807 08:37:53.879725 32714 ts_manager.cc:194] Registered new tserver with Master: 47fa071b785242bdb0861aa94d868819 (127.9.174.2:37327)
I20260807 08:37:53.880381 32714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:39815
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:37:53.900516 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43599
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:54.036190 756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:54.036521 756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:54.036569 756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:54.040692 756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:54.040756 756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:54.040817 756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:37:54.047207 756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:37:54.048735 756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:54.049988 756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:54.062044 762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:54.062184 756 server_base.cc:1034] running on GCE node
W20260807 08:37:54.062048 768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:54.062448 764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:54.062995 756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:54.063901 756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:54.065136 756 hybrid_clock.cc:648] HybridClock initialized: now 1786091874065099 us; error 51 us; skew 500 ppm
Aug 07 08:37:54 dist-test-slave-h3ft krb5kdc[32638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091874, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:54.068681 756 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:37:54.069888 756 webserver.cc:460] Webserver started at http://127.9.174.3:33341/ using document root <none> and password file <none>
I20260807 08:37:54.070600 756 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:54.070706 756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:54.070992 756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:54.073598 756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "fc6016cae0ab49b187362a8a6cea5acf"
format_stamp: "Formatted at 2026-08-07 08:37:54 on dist-test-slave-h3ft"
server_key: "6e9587a91159b439d72d5961a6b45b86"
server_key_iv: "654628d5a95d1c53eb95ed123e674640"
server_key_version: "encryptionkey@0"
I20260807 08:37:54.074285 756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "fc6016cae0ab49b187362a8a6cea5acf"
format_stamp: "Formatted at 2026-08-07 08:37:54 on dist-test-slave-h3ft"
server_key: "6e9587a91159b439d72d5961a6b45b86"
server_key_iv: "654628d5a95d1c53eb95ed123e674640"
server_key_version: "encryptionkey@0"
I20260807 08:37:54.079058 756 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260807 08:37:54.082135 782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:54.083774 756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:54.083933 756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "fc6016cae0ab49b187362a8a6cea5acf"
format_stamp: "Formatted at 2026-08-07 08:37:54 on dist-test-slave-h3ft"
server_key: "6e9587a91159b439d72d5961a6b45b86"
server_key_iv: "654628d5a95d1c53eb95ed123e674640"
server_key_version: "encryptionkey@0"
I20260807 08:37:54.084075 756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:54.120882 756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:54.125056 756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:54.125316 756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:54.126137 756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:54.127280 756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:54.127373 756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:54.127471 756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:54.127524 756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:54.138543 756 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:35549
I20260807 08:37:54.138572 958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:35549 every 8 connection(s)
I20260807 08:37:54.139761 756 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:37:54.140013 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 756
I20260807 08:37:54.140161 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:37:54.140614 9912 external_mini_cluster.cc:1428] Setting key 44bfad833b739e13fd07734b8c9e71ac
Aug 07 08:37:54 dist-test-slave-h3ft krb5kdc[32638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091874, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:37:54.157104 308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.142353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:49945 (local address 127.9.174.62:43599)
0807 08:37:54.142558 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.142563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.143850 (+ 1287us) server_negotiation.cc:407] Connection header received
0807 08:37:54.145730 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.145733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.145789 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.145862 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:54.147417 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.148016 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.149314 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.149558 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.149642 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.152769 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:54.152773 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:54.152775 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:54.152799 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:54.154253 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:54.154841 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:54.154846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:54.154848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:54.154951 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:54.155484 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:54.155488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:54.155489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:54.155772 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:54.155882 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:54.156709 (+ 827us) server_negotiation.cc:299] Negotiation successful
0807 08:37:54.156863 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 08:37:54.158143 961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.142901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:49945)
0807 08:37:54.143660 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.143729 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.145510 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.146062 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.146069 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.146595 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:54.147271 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.147289 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.148138 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.148141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.149192 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.149203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.149716 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.149719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.149895 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.150721 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:54.150740 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:54.152592 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:54.154375 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:54.154380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:54.154395 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:54.154705 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:54.155101 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:54.155104 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:54.155124 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:54.155357 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:54.155899 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:54.155909 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:54.156321 (+ 412us) client_negotiation.cc:769] Sending connection context
0807 08:37:54.156608 (+ 287us) client_negotiation.cc:240] Negotiation successful
0807 08:37:54.157043 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":199,"threads_started":1}
I20260807 08:37:54.159659 959 heartbeater.cc:347] Connected to a master server at 127.9.174.62:43599
I20260807 08:37:54.159984 959 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:54.160646 959 heartbeater.cc:511] Master 127.9.174.62:43599 requested a full tablet report, sending...
I20260807 08:37:54.161907 32714 ts_manager.cc:194] Registered new tserver with Master: fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549)
I20260807 08:37:54.162624 32714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:49945
I20260807 08:37:54.168948 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:37:54.179675 308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.171492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38820 (local address 127.9.174.62:43599)
0807 08:37:54.171660 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.171667 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.171880 (+ 213us) server_negotiation.cc:407] Connection header received
0807 08:37:54.172050 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.172054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.172110 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.172197 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:54.173007 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.173576 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.174464 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.174715 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.174826 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.176459 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:54.176463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:54.176466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:54.176500 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:54.177789 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:54.178242 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:54.178246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:54.178248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:54.178351 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:54.178640 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:54.178643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:54.178645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:54.178915 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:54.179092 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:54.179356 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 08:37:54.179488 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 08:37:54.183205 32714 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:54.187052 32714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:54.204489 977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.198143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:37327 (local address 127.0.0.1:43454)
0807 08:37:54.198529 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.198574 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.198753 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.199745 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.199751 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.199774 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:54.199958 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.199967 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.201312 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.201315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.202587 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.202597 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.203833 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.203837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.203966 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.204011 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 08:37:54.204138 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 08:37:54.204282 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":119,"threads_started":1}
I20260807 08:37:54.204505 978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.198353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43454 (local address 127.9.174.2:37327)
0807 08:37:54.198821 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.198839 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.198868 (+ 29us) server_negotiation.cc:407] Connection header received
0807 08:37:54.198965 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.198972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.199247 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.199451 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:54.200117 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.201133 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.202703 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.203698 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.203842 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.204065 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:54.204172 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 08:37:54.204295 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":199,"threads_started":1}
I20260807 08:37:54.205077 982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.199422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:46005 (local address 127.0.0.1:47756)
0807 08:37:54.200026 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.200041 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.200142 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.200999 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.201002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.201022 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:54.201163 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.201171 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.202598 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.202602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.203467 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.203478 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.204615 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.204618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.204779 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.204806 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 08:37:54.204861 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 08:37:54.204945 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":129,"threads_started":1}
I20260807 08:37:54.205552 980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.199036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:35549 (local address 127.0.0.1:36522)
0807 08:37:54.199379 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.199395 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.199509 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.200407 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.200411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.200432 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:54.200569 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.200576 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.202627 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.202631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.203843 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.203856 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.205095 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.205100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.205217 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.205246 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 08:37:54.205306 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 08:37:54.205393 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":121,"threads_started":1}
I20260807 08:37:54.205958 983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.199999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47756 (local address 127.9.174.1:46005)
0807 08:37:54.200463 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.200477 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.200506 (+ 29us) server_negotiation.cc:407] Connection header received
0807 08:37:54.200604 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.200610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.200814 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.201000 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:54.201293 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.202447 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.203610 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.204467 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.205189 (+ 722us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.205631 (+ 442us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:54.205688 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 08:37:54.205787 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":177,"threads_started":1}
I20260807 08:37:54.206383 981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.199297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36522 (local address 127.9.174.3:35549)
0807 08:37:54.199834 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.199851 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.199881 (+ 30us) server_negotiation.cc:407] Connection header received
0807 08:37:54.199984 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.199990 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.200211 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.200744 (+ 533us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:54.200963 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.202447 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.204013 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.204928 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.205860 (+ 932us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.206028 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:54.206084 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 08:37:54.206183 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":127,"threads_started":1}
I20260807 08:37:54.209197 644 tablet_service.cc:1467] Processing CreateTablet for tablet 70779139e6d644ee9da6c440fa7ee3cc (DEFAULT_TABLE table=test-table [id=89a48279518f49b59380160974ec66b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:54.209178 854 tablet_service.cc:1467] Processing CreateTablet for tablet 70779139e6d644ee9da6c440fa7ee3cc (DEFAULT_TABLE table=test-table [id=89a48279518f49b59380160974ec66b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:54.209178 437 tablet_service.cc:1467] Processing CreateTablet for tablet 70779139e6d644ee9da6c440fa7ee3cc (DEFAULT_TABLE table=test-table [id=89a48279518f49b59380160974ec66b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:54.210522 644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70779139e6d644ee9da6c440fa7ee3cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:54.210534 854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70779139e6d644ee9da6c440fa7ee3cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:54.210526 437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70779139e6d644ee9da6c440fa7ee3cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:54.218591 984 tablet_bootstrap.cc:492] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819: Bootstrap starting.
I20260807 08:37:54.219614 988 tablet_bootstrap.cc:492] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf: Bootstrap starting.
I20260807 08:37:54.219678 987 tablet_bootstrap.cc:492] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: Bootstrap starting.
I20260807 08:37:54.222134 988 tablet_bootstrap.cc:654] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:54.222134 987 tablet_bootstrap.cc:654] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:54.222440 984 tablet_bootstrap.cc:654] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:54.223313 984 log.cc:824] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:54.223320 987 log.cc:824] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:54.223546 988 log.cc:824] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:54.225497 984 tablet_bootstrap.cc:492] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819: No bootstrap required, opened a new log
I20260807 08:37:54.225499 987 tablet_bootstrap.cc:492] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: No bootstrap required, opened a new log
I20260807 08:37:54.225719 984 ts_tablet_manager.cc:1397] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 08:37:54.225723 987 ts_tablet_manager.cc:1397] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 08:37:54.228812 988 tablet_bootstrap.cc:492] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf: No bootstrap required, opened a new log
I20260807 08:37:54.229036 988 ts_tablet_manager.cc:1397] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260807 08:37:54.230043 987 raft_consensus.cc:348] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:37:54.230049 984 raft_consensus.cc:348] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:37:54.230331 984 raft_consensus.cc:374] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:54.230340 987 raft_consensus.cc:374] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:54.230409 987 raft_consensus.cc:729] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 757fe0884d894fa380b34c2928462b9f, State: Initialized, Role: FOLLOWER
I20260807 08:37:54.230423 984 raft_consensus.cc:729] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47fa071b785242bdb0861aa94d868819, State: Initialized, Role: FOLLOWER
I20260807 08:37:54.230967 984 consensus_queue.cc:260] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:37:54.230964 987 consensus_queue.cc:260] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:37:54.232206 984 ts_tablet_manager.cc:1428] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260807 08:37:54.232208 987 ts_tablet_manager.cc:1428] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 08:37:54.232285 744 heartbeater.cc:503] Master 127.9.174.62:43599 was elected leader, sending a full tablet report...
I20260807 08:37:54.232632 530 heartbeater.cc:503] Master 127.9.174.62:43599 was elected leader, sending a full tablet report...
I20260807 08:37:54.233248 988 raft_consensus.cc:348] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:37:54.233557 988 raft_consensus.cc:374] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:54.233628 988 raft_consensus.cc:729] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6016cae0ab49b187362a8a6cea5acf, State: Initialized, Role: FOLLOWER
I20260807 08:37:54.234135 988 consensus_queue.cc:260] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:37:54.234952 959 heartbeater.cc:503] Master 127.9.174.62:43599 was elected leader, sending a full tablet report...
I20260807 08:37:54.235373 988 ts_tablet_manager.cc:1428] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260807 08:37:54.325733 532 tablet.cc:2369] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:37:54.358101 745 tablet.cc:2369] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:37:54.391278 960 tablet.cc:2369] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:37:54.572252 998 raft_consensus.cc:482] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:54.572737 998 raft_consensus.cc:504] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:37:54.574352 998 leader_election.cc:288] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47fa071b785242bdb0861aa94d868819 (127.9.174.2:37327), 757fe0884d894fa380b34c2928462b9f (127.9.174.1:46005)
I20260807 08:37:54.577351 995 raft_consensus.cc:482] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:54.577569 995 raft_consensus.cc:504] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:37:54.578967 995 leader_election.cc:288] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549), 757fe0884d894fa380b34c2928462b9f (127.9.174.1:46005)
I20260807 08:37:54.579609 961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.574692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:37327 (local address 127.9.174.3:50695)
0807 08:37:54.574892 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.574913 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.575076 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.575549 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.575552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.575575 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:54.575725 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.575731 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.576862 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.576864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.578040 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.578052 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.579071 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.579076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.579199 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.579244 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 08:37:54.579319 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 08:37:54.579413 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 08:37:54.580211 983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.575340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:59727 (local address 127.9.174.1:46005)
0807 08:37:54.575497 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.575517 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.575720 (+ 203us) server_negotiation.cc:407] Connection header received
0807 08:37:54.575941 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.575945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.576009 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.576092 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:54.576424 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.577159 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.578745 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.579418 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.579524 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.579595 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:54.579944 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 08:37:54.580043 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 08:37:54.580543 978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.574797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:50695 (local address 127.9.174.2:37327)
0807 08:37:54.574989 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.575016 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.575040 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:37:54.575253 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.575257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.575321 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.575391 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:54.575840 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.576708 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.578194 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.578783 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.578891 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.578976 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:54.580309 (+ 1333us) server_negotiation.cc:299] Negotiation successful
0807 08:37:54.580414 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 08:37:54.580732 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.575058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:46005 (local address 127.9.174.3:59727)
0807 08:37:54.575614 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.575625 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.575762 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.576119 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.576122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.576142 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:54.576286 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.576293 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.577310 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.577313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.578596 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.578607 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.579715 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.579718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.579819 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.579840 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 08:37:54.580523 (+ 683us) client_negotiation.cc:240] Negotiation successful
0807 08:37:54.580595 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":209,"threads_started":1}
I20260807 08:37:54.581197 670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70779139e6d644ee9da6c440fa7ee3cc" candidate_uuid: "fc6016cae0ab49b187362a8a6cea5acf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47fa071b785242bdb0861aa94d868819" is_pre_election: true
I20260807 08:37:54.581377 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70779139e6d644ee9da6c440fa7ee3cc" candidate_uuid: "fc6016cae0ab49b187362a8a6cea5acf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "757fe0884d894fa380b34c2928462b9f" is_pre_election: true
I20260807 08:37:54.581466 670 raft_consensus.cc:2454] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc6016cae0ab49b187362a8a6cea5acf in term 0.
I20260807 08:37:54.581653 463 raft_consensus.cc:2454] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc6016cae0ab49b187362a8a6cea5acf in term 0.
I20260807 08:37:54.582269 790 leader_election.cc:302] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 757fe0884d894fa380b34c2928462b9f, fc6016cae0ab49b187362a8a6cea5acf; no voters:
I20260807 08:37:54.582877 998 raft_consensus.cc:2781] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:54.582971 998 raft_consensus.cc:482] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:54.583069 998 raft_consensus.cc:3037] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:54.584738 998 raft_consensus.cc:504] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:37:54.585191 998 leader_election.cc:288] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [CANDIDATE]: Term 1 election: Requested vote from peers 47fa071b785242bdb0861aa94d868819 (127.9.174.2:37327), 757fe0884d894fa380b34c2928462b9f (127.9.174.1:46005)
I20260807 08:37:54.585373 981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.580157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:50851 (local address 127.9.174.3:35549)
0807 08:37:54.580498 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.580503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.581064 (+ 561us) server_negotiation.cc:407] Connection header received
0807 08:37:54.581282 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.581287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.581340 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.581389 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:54.581816 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.582475 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.583977 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.584563 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.584659 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.584731 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:54.585179 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 08:37:54.585264 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260}
I20260807 08:37:54.585388 1000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.579357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:35549 (local address 127.9.174.2:50851)
0807 08:37:54.580976 (+ 1619us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.580991 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.581158 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.581547 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.581549 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.581571 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:54.581687 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.581695 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.582634 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.582637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.583831 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.583843 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.584724 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.584727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.585016 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.585044 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 08:37:54.585125 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 08:37:54.585217 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1548,"mutex_wait_us":138,"thread_start_us":133,"threads_started":1}
I20260807 08:37:54.585753 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70779139e6d644ee9da6c440fa7ee3cc" candidate_uuid: "fc6016cae0ab49b187362a8a6cea5acf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "757fe0884d894fa380b34c2928462b9f"
I20260807 08:37:54.585908 463 raft_consensus.cc:3037] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:54.586110 670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70779139e6d644ee9da6c440fa7ee3cc" candidate_uuid: "fc6016cae0ab49b187362a8a6cea5acf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47fa071b785242bdb0861aa94d868819"
I20260807 08:37:54.586243 670 raft_consensus.cc:3037] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:54.586496 881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70779139e6d644ee9da6c440fa7ee3cc" candidate_uuid: "47fa071b785242bdb0861aa94d868819" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6016cae0ab49b187362a8a6cea5acf" is_pre_election: true
I20260807 08:37:54.586755 881 raft_consensus.cc:2379] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 47fa071b785242bdb0861aa94d868819 in current term 1: Already voted for candidate fc6016cae0ab49b187362a8a6cea5acf in this term.
I20260807 08:37:54.586812 983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.580039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:36037 (local address 127.9.174.1:46005)
0807 08:37:54.580392 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.580396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.582427 (+ 2031us) server_negotiation.cc:407] Connection header received
0807 08:37:54.582610 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.582613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.582651 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.582693 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:54.583080 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.583807 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.585251 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.585842 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.585932 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.585992 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:54.586610 (+ 618us) server_negotiation.cc:299] Negotiation successful
0807 08:37:54.586687 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":138,"threads_started":1}
I20260807 08:37:54.587504 463 raft_consensus.cc:2454] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc6016cae0ab49b187362a8a6cea5acf in term 1.
I20260807 08:37:54.587541 1001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.579777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:46005 (local address 127.9.174.2:36037)
0807 08:37:54.582333 (+ 2556us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.582348 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.582503 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.582808 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.582810 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.582824 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:54.582938 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.582943 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.583946 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.583949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.585121 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.585131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.586101 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.586104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.586461 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.586488 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 08:37:54.586934 (+ 446us) client_negotiation.cc:240] Negotiation successful
0807 08:37:54.587329 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2485,"mutex_wait_us":33,"thread_start_us":81,"threads_started":1}
I20260807 08:37:54.587919 670 raft_consensus.cc:2454] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc6016cae0ab49b187362a8a6cea5acf in term 1.
I20260807 08:37:54.587900 790 leader_election.cc:302] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 757fe0884d894fa380b34c2928462b9f, fc6016cae0ab49b187362a8a6cea5acf; no voters:
I20260807 08:37:54.588119 998 raft_consensus.cc:2781] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:54.588088 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70779139e6d644ee9da6c440fa7ee3cc" candidate_uuid: "47fa071b785242bdb0861aa94d868819" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "757fe0884d894fa380b34c2928462b9f" is_pre_election: true
I20260807 08:37:54.588302 463 raft_consensus.cc:2379] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 47fa071b785242bdb0861aa94d868819 in current term 1: Already voted for candidate fc6016cae0ab49b187362a8a6cea5acf in this term.
I20260807 08:37:54.588321 998 raft_consensus.cc:686] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [term 1 LEADER]: Becoming Leader. State: Replica: fc6016cae0ab49b187362a8a6cea5acf, State: Running, Role: LEADER
I20260807 08:37:54.588754 998 consensus_queue.cc:237] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:37:54.588851 567 leader_election.cc:302] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 47fa071b785242bdb0861aa94d868819; no voters: 757fe0884d894fa380b34c2928462b9f, fc6016cae0ab49b187362a8a6cea5acf
I20260807 08:37:54.589159 995 raft_consensus.cc:2726] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 08:37:54.592928 32707 catalog_manager.cc:5495] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf reported cstate change: term changed from 0 to 1, leader changed from <none> to fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3). New cstate: current_term: 1 leader_uuid: "fc6016cae0ab49b187362a8a6cea5acf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } health_report { overall_health: UNKNOWN } } }
I20260807 08:37:54.663384 981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.659059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36528 (local address 127.9.174.3:35549)
0807 08:37:54.659244 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.659251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.659268 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:37:54.659385 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.659388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.659464 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.659604 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:54.659930 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.660495 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.661491 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.661740 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.661864 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.662256 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:54.662844 (+ 588us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:54.663006 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:54.663115 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 08:37:54.663191 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 08:37:54.671816 463 raft_consensus.cc:1261] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Refusing update from remote peer fc6016cae0ab49b187362a8a6cea5acf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:54.671866 670 raft_consensus.cc:1261] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819 [term 1 FOLLOWER]: Refusing update from remote peer fc6016cae0ab49b187362a8a6cea5acf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:54.672665 998 consensus_queue.cc:1035] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [LEADER]: Connected to new peer: Peer: permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:54.672991 1003 consensus_queue.cc:1035] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [LEADER]: Connected to new peer: Peer: permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:54.683789 1014 mvcc.cc:205] Tried to move back new op lower bound from 7315832318644875264 to 7315832318318911488. Current Snapshot: MvccSnapshot[applied={T|T < 7315832318644875264}]
I20260807 08:37:54.697463 32707 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:54.697786 32707 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:54.701849 32707 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Sending DeleteTablet for 3 replicas of tablet 70779139e6d644ee9da6c440fa7ee3cc
I20260807 08:37:54.703398 437 tablet_service.cc:1514] Processing DeleteTablet for tablet 70779139e6d644ee9da6c440fa7ee3cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:54 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:47756
I20260807 08:37:54.704234 1026 tablet_replica.cc:330] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: stopping tablet replica
I20260807 08:37:54.704773 644 tablet_service.cc:1514] Processing DeleteTablet for tablet 70779139e6d644ee9da6c440fa7ee3cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:54 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:43454
I20260807 08:37:54.704871 1026 raft_consensus.cc:2229] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:54.705060 854 tablet_service.cc:1514] Processing DeleteTablet for tablet 70779139e6d644ee9da6c440fa7ee3cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:54 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:36522
I20260807 08:37:54.705650 1026 raft_consensus.cc:2258] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:54.706256 1029 tablet_replica.cc:330] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf: stopping tablet replica
I20260807 08:37:54.706574 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 340
I20260807 08:37:54.706619 1029 raft_consensus.cc:2229] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:37:54.710394 1029 raft_consensus.cc:2258] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:54.716249 1029 ts_tablet_manager.cc:1905] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:54.719846 1029 ts_tablet_manager.cc:1918] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:54.720042 1029 log.cc:1196] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/70779139e6d644ee9da6c440fa7ee3cc
I20260807 08:37:54.720506 1029 ts_tablet_manager.cc:1939] T 70779139e6d644ee9da6c440fa7ee3cc P fc6016cae0ab49b187362a8a6cea5acf: Deleting consensus metadata
I20260807 08:37:54.723326 32686 catalog_manager.cc:4841] TS fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549): tablet 70779139e6d644ee9da6c440fa7ee3cc (table test-table [id=89a48279518f49b59380160974ec66b3]) successfully deleted
I20260807 08:37:54.727432 1028 tablet_replica.cc:330] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819: stopping tablet replica
I20260807 08:37:54.727953 1028 raft_consensus.cc:2229] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:54.728466 1028 raft_consensus.cc:2258] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:54.730435 1028 ts_tablet_manager.cc:1905] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:54.733723 1028 ts_tablet_manager.cc:1918] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:54.733875 1028 log.cc:1196] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/70779139e6d644ee9da6c440fa7ee3cc
I20260807 08:37:54.734205 1028 ts_tablet_manager.cc:1939] T 70779139e6d644ee9da6c440fa7ee3cc P 47fa071b785242bdb0861aa94d868819: Deleting consensus metadata
I20260807 08:37:54.736037 32686 catalog_manager.cc:4841] TS 47fa071b785242bdb0861aa94d868819 (127.9.174.2:37327): tablet 70779139e6d644ee9da6c440fa7ee3cc (table test-table [id=89a48279518f49b59380160974ec66b3]) successfully deleted
W20260807 08:37:54.736410 32686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:46005 (error 108)
I20260807 08:37:54.736697 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 540
I20260807 08:37:54.751163 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.750135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:46005 (local address 127.0.0.1:47768)
0807 08:37:54.750800 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.750937 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:46005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":426,"threads_started":1}
W20260807 08:37:54.751649 32686 catalog_manager.cc:4584] TS 757fe0884d894fa380b34c2928462b9f (127.9.174.1:46005): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 70779139e6d644ee9da6c440fa7ee3cc: Network error: Client connection negotiation failed: client connection to 127.9.174.1:46005: connect: Connection refused (error 111)
I20260807 08:37:54.765298 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 756
I20260807 08:37:54.781504 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 32664
I20260807 08:38:14.798903 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:43599
--webserver_interface=127.9.174.62
--webserver_port=44787
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:43599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:38:14.941155 1032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:14.941532 1032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:14.941581 1032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:14.946092 1032 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 08:38:14.946187 1032 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 08:38:14.946204 1032 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 08:38:14.946219 1032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:38:14.946239 1032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:14.946259 1032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:14.946323 1032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:38:14.946352 1032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:38:14.952502 1032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:43599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:43599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=44787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:14.954015 1032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:14.955374 1032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:14.972873 1040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:14.972929 1032 server_base.cc:1034] running on GCE node
W20260807 08:38:14.972873 1047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:14.972873 1044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:14.974108 1032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:14.975632 1032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:14.976832 1032 hybrid_clock.cc:648] HybridClock initialized: now 1786091894976790 us; error 59 us; skew 500 ppm
Aug 07 08:38:14 dist-test-slave-h3ft krb5kdc[32638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091894, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:14.984912 1032 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:38:14.986193 1032 webserver.cc:460] Webserver started at http://127.9.174.62:44787/ using document root <none> and password file <none>
I20260807 08:38:14.986967 1032 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:14.987147 1032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:14.992293 1032 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:38:14.995239 1060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:14.996872 1032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:38:14.997066 1032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "bbeaeef13c814174b459f3f7eedd66b2"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h3ft"
server_key: "fa11fb4127875411ce1036c3dbd720d0"
server_key_iv: "e263d3838e4a75220cebdafbca95a662"
server_key_version: "encryptionkey@0"
I20260807 08:38:14.997879 1032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:15.014587 1032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:15.018879 1032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:15.019214 1032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:15.029032 1032 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:43599
I20260807 08:38:15.029435 1145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:43599 every 8 connection(s)
I20260807 08:38:15.030684 1032 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:38:15.036749 1146 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:38:15.038971 1146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Bootstrap starting.
I20260807 08:38:15.040395 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 1032
I20260807 08:38:15.040978 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:46005
--local_ip_for_outbound_sockets=127.9.174.1
--tserver_master_addrs=127.9.174.62:43599
--webserver_port=40153
--webserver_interface=127.9.174.1
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:38:15.063479 1146 log.cc:824] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:15.074821 1146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:38:15.075450 1146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Bootstrap complete.
I20260807 08:38:15.080751 1146 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } }
I20260807 08:38:15.081316 1146 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbeaeef13c814174b459f3f7eedd66b2, State: Initialized, Role: FOLLOWER
I20260807 08:38:15.081966 1146 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } }
I20260807 08:38:15.082152 1146 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:38:15.082268 1146 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:38:15.082417 1146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:38:15.085047 1146 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } }
I20260807 08:38:15.085490 1146 leader_election.cc:302] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbeaeef13c814174b459f3f7eedd66b2; no voters:
I20260807 08:38:15.085949 1146 leader_election.cc:288] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 08:38:15.086076 1152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:38:15.086372 1152 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 2 LEADER]: Becoming Leader. State: Replica: bbeaeef13c814174b459f3f7eedd66b2, State: Running, Role: LEADER
I20260807 08:38:15.086829 1152 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } }
I20260807 08:38:15.087603 1146 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:38:15.088762 1156 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbeaeef13c814174b459f3f7eedd66b2. Latest consensus state: current_term: 2 leader_uuid: "bbeaeef13c814174b459f3f7eedd66b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } } }
I20260807 08:38:15.088809 1154 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bbeaeef13c814174b459f3f7eedd66b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } } }
I20260807 08:38:15.088964 1154 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:15.088910 1156 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:15.094637 1180 catalog_manager.cc:1259] Loaded cluster ID: 2a5bb6cefc8141aa9b5f8e6a47fdb26b
I20260807 08:38:15.094738 1180 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: loading cluster ID for follower catalog manager: success
I20260807 08:38:15.098265 1180 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: acquiring CA information for follower catalog manager: success
I20260807 08:38:15.100379 1180 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:38:15.100611 1170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:38:15.103646 1170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:38:15.104209 1170 catalog_manager.cc:1259] Loaded cluster ID: 2a5bb6cefc8141aa9b5f8e6a47fdb26b
I20260807 08:38:15.104281 1170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:38:15.105049 1170 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:38:15.105561 1170 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Loaded TSK: 0
I20260807 08:38:15.121021 1170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Generated new TSK 1
I20260807 08:38:15.121414 1170 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:38:15.198108 1149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:15.198433 1149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:15.198485 1149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:15.202930 1149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:15.203058 1149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:15.203186 1149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:38:15.208880 1149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:46005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=40153
--tserver_master_addrs=127.9.174.62:43599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:15.210175 1149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:15.211493 1149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:15.222913 1194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:15.223165 1196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:15.222910 1193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:15.223371 1149 server_base.cc:1034] running on GCE node
I20260807 08:38:15.223827 1149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:15.224637 1149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:15.225827 1149 hybrid_clock.cc:648] HybridClock initialized: now 1786091895225797 us; error 48 us; skew 500 ppm
Aug 07 08:38:15 dist-test-slave-h3ft krb5kdc[32638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091895, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:15.229447 1149 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:38:15.230490 1149 webserver.cc:460] Webserver started at http://127.9.174.1:40153/ using document root <none> and password file <none>
I20260807 08:38:15.231262 1149 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:15.231382 1149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:15.236529 1149 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:38:15.239408 1209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:15.241110 1149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:38:15.241278 1149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "757fe0884d894fa380b34c2928462b9f"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h3ft"
server_key: "43fdd10fa3b1658ba39cf0150d663ca7"
server_key_iv: "7d542f8083822e3778e5dc39b0eae33d"
server_key_version: "encryptionkey@0"
I20260807 08:38:15.242077 1149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:15.263715 1149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:15.267829 1149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:15.268096 1149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:15.268903 1149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:15.270474 1223 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:38:15.276165 1149 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:38:15.276276 1149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260807 08:38:15.276427 1149 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:38:15.278568 1149 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:38:15.278645 1149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 08:38:15.279083 1223 tablet_bootstrap.cc:492] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: Bootstrap starting.
I20260807 08:38:15.291352 1149 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:46005
I20260807 08:38:15.291610 1389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:46005 every 8 connection(s)
I20260807 08:38:15.292564 1149 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:38:15.302627 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 1149
I20260807 08:38:15.303355 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:37327
--local_ip_for_outbound_sockets=127.9.174.2
--tserver_master_addrs=127.9.174.62:43599
--webserver_port=42187
--webserver_interface=127.9.174.2
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:38:15.304963 1223 log.cc:824] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:38:15 dist-test-slave-h3ft krb5kdc[32638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091895, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:38:15.314102 1397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.295580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:49609)
0807 08:38:15.296685 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:15.296760 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:15.298200 (+ 1440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:15.299754 (+ 1554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:15.299772 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:15.300525 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:15.301265 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:15.301289 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.303166 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.303169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:15.304394 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:15.304405 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.305294 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.305298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:15.305547 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.306349 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:15.306385 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:15.308199 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:15.310586 (+ 2387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:15.310592 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:15.310607 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:15.310879 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:15.311286 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:15.311291 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:15.311293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:15.311447 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:15.312104 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:15.312110 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:15.312379 (+ 269us) client_negotiation.cc:769] Sending connection context
0807 08:38:15.312573 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 08:38:15.312983 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":178,"threads_started":1}
I20260807 08:38:15.314085 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.295572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:49609 (local address 127.9.174.62:43599)
0807 08:38:15.296189 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:15.296216 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:15.296942 (+ 726us) server_negotiation.cc:407] Connection header received
0807 08:38:15.298775 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:15.298786 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:15.299243 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:15.299579 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:15.301543 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.302851 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.304531 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.305039 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.305250 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.308432 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:15.308438 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:15.308452 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:15.308484 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:15.310447 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:15.311033 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:15.311042 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:15.311052 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:15.311181 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:15.311547 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:15.311552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:15.311553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:15.311974 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:15.312122 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:15.312727 (+ 605us) server_negotiation.cc:299] Negotiation successful
0807 08:38:15.313033 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":199,"threads_started":1}
I20260807 08:38:15.317116 1223 tablet_bootstrap.cc:492] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:38:15.317152 1391 heartbeater.cc:347] Connected to a master server at 127.9.174.62:43599
I20260807 08:38:15.317394 1391 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:15.317677 1223 tablet_bootstrap.cc:492] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: Bootstrap complete.
I20260807 08:38:15.317922 1391 heartbeater.cc:511] Master 127.9.174.62:43599 requested a full tablet report, sending...
I20260807 08:38:15.319710 1223 ts_tablet_manager.cc:1397] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: Time spent bootstrapping tablet: real 0.041s user 0.029s sys 0.010s
I20260807 08:38:15.320830 1084 ts_manager.cc:194] Registered new tserver with Master: 757fe0884d894fa380b34c2928462b9f (127.9.174.1:46005)
I20260807 08:38:15.323060 1223 raft_consensus.cc:348] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:15.323650 1223 raft_consensus.cc:729] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 757fe0884d894fa380b34c2928462b9f, State: Initialized, Role: FOLLOWER
I20260807 08:38:15.324198 1223 consensus_queue.cc:260] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:15.325301 1223 ts_tablet_manager.cc:1428] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 08:38:15.325461 1084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:49609
I20260807 08:38:15.328667 1391 heartbeater.cc:503] Master 127.9.174.62:43599 was elected leader, sending a full tablet report...
I20260807 08:38:15.332631 1403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.324146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55380 (local address 127.9.174.1:46005)
0807 08:38:15.326446 (+ 2300us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:15.326457 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:15.326917 (+ 460us) server_negotiation.cc:407] Connection header received
0807 08:38:15.327192 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:15.327198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:15.327570 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:15.327738 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:15.328089 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.329134 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.330832 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.331884 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.332016 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.332181 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:15.332369 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 08:38:15.332463 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2191,"thread_start_us":155,"threads_started":1}
I20260807 08:38:15.332641 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.324097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:46005 (local address 127.0.0.1:55380)
0807 08:38:15.326739 (+ 2642us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:15.326792 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:15.326961 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:15.327761 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:15.327768 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:15.327798 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:15.327953 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:15.327960 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.329254 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.329257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:15.330661 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:15.330673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.332021 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.332025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:15.332149 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.332189 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 08:38:15.332344 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 08:38:15.332490 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2433,"thread_start_us":155,"threads_started":1}
I20260807 08:38:15.333873 1292 tablet_service.cc:1514] Processing DeleteTablet for tablet 70779139e6d644ee9da6c440fa7ee3cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:54 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:55380
I20260807 08:38:15.334882 1412 tablet_replica.cc:330] stopping tablet replica
I20260807 08:38:15.335332 1412 raft_consensus.cc:2229] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:38:15.335646 1412 raft_consensus.cc:2258] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:38:15.336952 1412 ts_tablet_manager.cc:1905] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:38:15.342346 1412 ts_tablet_manager.cc:1918] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:38:15.342466 1412 log.cc:1196] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal/wals/70779139e6d644ee9da6c440fa7ee3cc
I20260807 08:38:15.342767 1412 ts_tablet_manager.cc:1939] T 70779139e6d644ee9da6c440fa7ee3cc P 757fe0884d894fa380b34c2928462b9f: Deleting consensus metadata
I20260807 08:38:15.345180 1064 catalog_manager.cc:4841] TS 757fe0884d894fa380b34c2928462b9f (127.9.174.1:46005): tablet 70779139e6d644ee9da6c440fa7ee3cc (table test-table [id=89a48279518f49b59380160974ec66b3]) successfully deleted
W20260807 08:38:15.451607 1400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:15.451985 1400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:15.452036 1400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:15.456372 1400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:15.456449 1400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:15.456516 1400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:38:15.462277 1400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:37327
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=42187
--tserver_master_addrs=127.9.174.62:43599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:15.463742 1400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:15.464979 1400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:15.477155 1419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:15.477144 1423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:15.477142 1420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:15.477609 1400 server_base.cc:1034] running on GCE node
I20260807 08:38:15.478078 1400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:15.478852 1400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:15.480043 1400 hybrid_clock.cc:648] HybridClock initialized: now 1786091895480011 us; error 50 us; skew 500 ppm
Aug 07 08:38:15 dist-test-slave-h3ft krb5kdc[32638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091895, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:15.484586 1400 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:38:15.485781 1400 webserver.cc:460] Webserver started at http://127.9.174.2:42187/ using document root <none> and password file <none>
I20260807 08:38:15.486521 1400 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:15.486637 1400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:15.491744 1400 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:38:15.494470 1430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:15.496133 1400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:38:15.496258 1400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "47fa071b785242bdb0861aa94d868819"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h3ft"
server_key: "029cb0f1a7c134157aef0aa0919bddc3"
server_key_iv: "6b84a0d3876d478ed778d5b4bf95bff0"
server_key_version: "encryptionkey@0"
I20260807 08:38:15.496927 1400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:15.512694 1400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:15.516489 1400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:15.516688 1400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:15.517403 1400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:15.518414 1400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:15.518466 1400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:15.518513 1400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:15.518529 1400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:15.529836 1400 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:37327
I20260807 08:38:15.529857 1591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:37327 every 8 connection(s)
I20260807 08:38:15.531183 1400 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:38:15.535372 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 1400
I20260807 08:38:15.535964 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:35549
--local_ip_for_outbound_sockets=127.9.174.3
--tserver_master_addrs=127.9.174.62:43599
--webserver_port=33341
--webserver_interface=127.9.174.3
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:38:15 dist-test-slave-h3ft krb5kdc[32638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091895, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:38:15.547904 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.533756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:54269 (local address 127.9.174.62:43599)
0807 08:38:15.533978 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:15.533983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:15.535036 (+ 1053us) server_negotiation.cc:407] Connection header received
0807 08:38:15.536805 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:15.536810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:15.536896 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:15.537009 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:15.538459 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.539200 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.540457 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.540718 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.540843 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.543833 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:15.543837 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:15.543841 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:15.543872 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:15.545554 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:15.546106 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:15.546111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:15.546113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:15.546204 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:15.546615 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:15.546620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:15.546622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:15.546830 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:15.546948 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:15.547543 (+ 595us) server_negotiation.cc:299] Negotiation successful
0807 08:38:15.547699 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 08:38:15.548938 1596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.534189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:54269)
0807 08:38:15.534781 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:15.534832 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:15.536540 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:15.537228 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:15.537237 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:15.537685 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:15.538284 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:15.538299 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.539341 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.539346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:15.540318 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:15.540328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.540910 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.540915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:15.541110 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.541907 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:15.541925 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:15.543627 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:15.545681 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:15.545687 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:15.545704 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:15.545998 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:15.546308 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:15.546311 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:15.546313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:15.546512 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:15.546947 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:15.546955 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:15.547283 (+ 328us) client_negotiation.cc:769] Sending connection context
0807 08:38:15.547503 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 08:38:15.547894 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":150,"threads_started":1}
I20260807 08:38:15.550541 1592 heartbeater.cc:347] Connected to a master server at 127.9.174.62:43599
I20260807 08:38:15.550901 1592 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:15.551584 1592 heartbeater.cc:511] Master 127.9.174.62:43599 requested a full tablet report, sending...
I20260807 08:38:15.553077 1084 ts_manager.cc:194] Registered new tserver with Master: 47fa071b785242bdb0861aa94d868819 (127.9.174.2:37327)
I20260807 08:38:15.553766 1084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:54269
W20260807 08:38:15.674912 1599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:15.675333 1599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:15.675431 1599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:15.680280 1599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:15.680425 1599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:15.680539 1599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:38:15.686905 1599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:35549
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=33341
--tserver_master_addrs=127.9.174.62:43599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:15.688907 1599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:15.690646 1599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:15.704221 1607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:15.704169 1606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:15.704130 1609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:15.704535 1599 server_base.cc:1034] running on GCE node
I20260807 08:38:15.705271 1599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:15.706208 1599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:15.707420 1599 hybrid_clock.cc:648] HybridClock initialized: now 1786091895707407 us; error 31 us; skew 500 ppm
Aug 07 08:38:15 dist-test-slave-h3ft krb5kdc[32638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091895, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:15.711380 1599 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:38:15.712805 1599 webserver.cc:460] Webserver started at http://127.9.174.3:33341/ using document root <none> and password file <none>
I20260807 08:38:15.713594 1599 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:15.713730 1599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:15.719652 1599 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:38:15.722541 1619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:15.724284 1599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:38:15.724483 1599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "fc6016cae0ab49b187362a8a6cea5acf"
format_stamp: "Formatted at 2026-08-07 08:37:54 on dist-test-slave-h3ft"
server_key: "6e9587a91159b439d72d5961a6b45b86"
server_key_iv: "654628d5a95d1c53eb95ed123e674640"
server_key_version: "encryptionkey@0"
I20260807 08:38:15.725275 1599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:15.753700 1599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:15.757721 1599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:15.757982 1599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:15.758771 1599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:15.759835 1599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:15.759912 1599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:15.759999 1599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:15.760058 1599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:15.771802 1599 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:35549
I20260807 08:38:15.771822 1794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:35549 every 8 connection(s)
I20260807 08:38:15.773200 1599 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:38:15.776731 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 1599
Aug 07 08:38:15 dist-test-slave-h3ft krb5kdc[32638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091895, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:38:15.786593 1799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.777589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33412 (local address 127.9.174.62:43599)
0807 08:38:15.777881 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:15.777886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:15.778111 (+ 225us) server_negotiation.cc:407] Connection header received
0807 08:38:15.778305 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:15.778308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:15.778370 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:15.778491 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:15.779365 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.780000 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.780902 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.781153 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.781271 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.783081 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:15.783084 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:15.783086 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:15.783107 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:15.784609 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:15.785139 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:15.785142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:15.785143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:15.785236 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:15.785559 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:15.785561 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:15.785562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:15.785776 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:15.786082 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:15.786178 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 08:38:15.786372 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":118,"threads_started":1}
I20260807 08:38:15.790046 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.775466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:33649 (local address 127.9.174.62:43599)
0807 08:38:15.775667 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:15.775672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:15.776720 (+ 1048us) server_negotiation.cc:407] Connection header received
0807 08:38:15.778661 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:15.778664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:15.778718 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:15.778797 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:15.780214 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.780739 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.781969 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.782265 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.782376 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.785645 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:15.785649 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:15.785652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:15.785679 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:15.787575 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:15.788190 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:15.788193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:15.788194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:15.788364 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:15.788805 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:15.788807 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:15.788808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:15.789081 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:15.789176 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:15.789760 (+ 584us) server_negotiation.cc:299] Negotiation successful
0807 08:38:15.789888 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 08:38:15.791225 1798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.776001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:33649)
0807 08:38:15.776546 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:15.776597 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:15.778428 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:15.779044 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:15.779056 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:15.779465 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:15.780058 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:15.780074 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.780880 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.780884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:15.781834 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:15.781844 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.782479 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.782486 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:15.782744 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.783494 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:15.783520 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:15.785475 (+ 1955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:15.787699 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:15.787706 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:15.787726 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:15.788094 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:15.788491 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:15.788494 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:15.788496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:15.788704 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:15.789188 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:15.789195 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:15.789493 (+ 298us) client_negotiation.cc:769] Sending connection context
0807 08:38:15.789717 (+ 224us) client_negotiation.cc:240] Negotiation successful
0807 08:38:15.790143 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":141,"threads_started":1}
I20260807 08:38:15.792462 1795 heartbeater.cc:347] Connected to a master server at 127.9.174.62:43599
I20260807 08:38:15.792794 1795 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:15.793437 1795 heartbeater.cc:511] Master 127.9.174.62:43599 requested a full tablet report, sending...
I20260807 08:38:15.794778 1084 ts_manager.cc:194] Registered new tserver with Master: fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549)
I20260807 08:38:15.795485 1084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:33649
I20260807 08:38:15.801146 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:38:15.805666 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.801908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33420 (local address 127.9.174.62:43599)
0807 08:38:15.802066 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:15.802070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:15.802399 (+ 329us) server_negotiation.cc:407] Connection header received
0807 08:38:15.802519 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:15.802521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:15.802655 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:15.802760 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:38:15.803061 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.803654 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.804528 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.804746 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.804824 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.805118 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:38:15.805298 (+ 180us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 08:38:15.805538 (+ 240us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33420: token expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 08:38:15.805768 1395 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33420: token expired
W20260807 08:38:15.806165 962 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.174.62:43599)
I20260807 08:38:15.814530 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.806636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33426 (local address 127.9.174.62:43599)
0807 08:38:15.806752 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:15.806755 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:15.806766 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:38:15.806798 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:15.806800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:15.806837 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:15.806903 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:15.807638 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.808150 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.809060 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.809284 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.809372 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.810936 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:15.810940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:15.810942 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:15.810974 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:15.812711 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:15.813167 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:15.813170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:15.813171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:15.813257 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:15.813597 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:15.813599 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:15.813600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:15.813855 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:15.813972 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:15.814243 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 08:38:15.814359 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 08:38:15.818053 1084 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:38:15.822286 1084 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:38:15.832187 1292 tablet_service.cc:1467] Processing CreateTablet for tablet 1f805244abd14d8fb280947432f7ac95 (DEFAULT_TABLE table=test-table [id=aaabf243f9a64feaaddb23c25c2d6d1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:38:15.833112 1292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f805244abd14d8fb280947432f7ac95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:15.836771 1810 tablet_bootstrap.cc:492] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f: Bootstrap starting.
I20260807 08:38:15.837359 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.830662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:35549 (local address 127.0.0.1:47924)
0807 08:38:15.830818 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:15.830838 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:15.830933 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:15.832363 (+ 1430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:15.832366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:15.832388 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:15.832525 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:15.832532 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.834256 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.834259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:15.835584 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:15.835596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.836858 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.836865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:15.836972 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.837006 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 08:38:15.837076 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 08:38:15.837179 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":102}
I20260807 08:38:15.837606 1807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.830947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:37327 (local address 127.0.0.1:46250)
0807 08:38:15.831244 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:15.831260 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:15.831365 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:15.832690 (+ 1325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:15.832693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:15.832712 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:15.832810 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:15.832816 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.834106 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.834111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:15.835584 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:15.835596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.837183 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.837186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:15.837275 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.837295 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 08:38:15.837355 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 08:38:15.837441 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":28,"thread_start_us":157,"threads_started":1}
I20260807 08:38:15.838009 1810 tablet_bootstrap.cc:654] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:15.838330 1808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.831210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47924 (local address 127.9.174.3:35549)
0807 08:38:15.831753 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:15.831769 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:15.831797 (+ 28us) server_negotiation.cc:407] Connection header received
0807 08:38:15.831894 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:15.831901 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:15.832175 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:15.832753 (+ 578us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:15.832971 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.834140 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.835744 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.836702 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.837668 (+ 966us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.837969 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:15.838033 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 08:38:15.838149 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":127,"threads_started":1}
I20260807 08:38:15.838330 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.831211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46250 (local address 127.9.174.2:37327)
0807 08:38:15.831837 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:15.831871 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:15.831899 (+ 28us) server_negotiation.cc:407] Connection header received
0807 08:38:15.831994 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:15.832000 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:15.832230 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:15.832416 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:15.832931 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.833969 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.835744 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.836702 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.837767 (+ 1065us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.837969 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:15.838033 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 08:38:15.838147 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":136,"threads_started":1}
I20260807 08:38:15.840085 1810 tablet_bootstrap.cc:492] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f: No bootstrap required, opened a new log
I20260807 08:38:15.840214 1810 ts_tablet_manager.cc:1397] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 08:38:15.840680 1810 raft_consensus.cc:348] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:15.840823 1810 raft_consensus.cc:374] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:15.840875 1810 raft_consensus.cc:729] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 757fe0884d894fa380b34c2928462b9f, State: Initialized, Role: FOLLOWER
I20260807 08:38:15.841034 1810 consensus_queue.cc:260] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:15.841431 1810 ts_tablet_manager.cc:1428] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260807 08:38:15.842332 1697 tablet_service.cc:1467] Processing CreateTablet for tablet 1f805244abd14d8fb280947432f7ac95 (DEFAULT_TABLE table=test-table [id=aaabf243f9a64feaaddb23c25c2d6d1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:38:15.842337 1502 tablet_service.cc:1467] Processing CreateTablet for tablet 1f805244abd14d8fb280947432f7ac95 (DEFAULT_TABLE table=test-table [id=aaabf243f9a64feaaddb23c25c2d6d1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:38:15.843861 1502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f805244abd14d8fb280947432f7ac95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:15.843865 1697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f805244abd14d8fb280947432f7ac95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:15.854059 1817 tablet_bootstrap.cc:492] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf: Bootstrap starting.
I20260807 08:38:15.854648 1816 tablet_bootstrap.cc:492] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819: Bootstrap starting.
I20260807 08:38:15.857321 1817 tablet_bootstrap.cc:654] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:15.857959 1816 tablet_bootstrap.cc:654] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:15.859076 1817 log.cc:824] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:15.859074 1816 log.cc:824] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:15.862193 1816 tablet_bootstrap.cc:492] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819: No bootstrap required, opened a new log
I20260807 08:38:15.862193 1817 tablet_bootstrap.cc:492] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf: No bootstrap required, opened a new log
I20260807 08:38:15.862465 1816 ts_tablet_manager.cc:1397] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 08:38:15.862465 1817 ts_tablet_manager.cc:1397] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 08:38:15.867293 1816 raft_consensus.cc:348] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:15.867293 1817 raft_consensus.cc:348] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:15.867697 1817 raft_consensus.cc:374] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:15.867697 1816 raft_consensus.cc:374] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:15.867780 1816 raft_consensus.cc:729] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47fa071b785242bdb0861aa94d868819, State: Initialized, Role: FOLLOWER
I20260807 08:38:15.867779 1817 raft_consensus.cc:729] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6016cae0ab49b187362a8a6cea5acf, State: Initialized, Role: FOLLOWER
I20260807 08:38:15.868397 1817 consensus_queue.cc:260] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:15.868396 1816 consensus_queue.cc:260] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:15.869621 1795 heartbeater.cc:503] Master 127.9.174.62:43599 was elected leader, sending a full tablet report...
I20260807 08:38:15.869854 1817 ts_tablet_manager.cc:1428] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf: Time spent starting tablet: real 0.007s user 0.009s sys 0.000s
I20260807 08:38:15.869854 1816 ts_tablet_manager.cc:1428] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819: Time spent starting tablet: real 0.007s user 0.009s sys 0.000s
I20260807 08:38:15.870239 1592 heartbeater.cc:503] Master 127.9.174.62:43599 was elected leader, sending a full tablet report...
I20260807 08:38:15.933060 1822 raft_consensus.cc:482] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:38:15.933403 1822 raft_consensus.cc:504] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:15.935104 1822 leader_election.cc:288] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549), 757fe0884d894fa380b34c2928462b9f (127.9.174.1:46005)
I20260807 08:38:15.940100 1808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.935551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:38599 (local address 127.9.174.3:35549)
0807 08:38:15.935726 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:15.935730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:15.935769 (+ 39us) server_negotiation.cc:407] Connection header received
0807 08:38:15.935943 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:15.935947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:15.936012 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:15.936123 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:15.936511 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.937321 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.938710 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.939456 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.939568 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.939641 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:15.939905 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 08:38:15.939975 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 08:38:15.940107 1596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.935433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:35549 (local address 127.9.174.2:38599)
0807 08:38:15.935620 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:15.935643 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:15.935774 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:15.936150 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:15.936154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:15.936177 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:15.936337 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:15.936347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.937452 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.937456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:15.938559 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:15.938569 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.939591 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.939595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:15.939722 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.939752 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 08:38:15.939834 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:38:15.939919 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":1}
I20260807 08:38:15.940840 1824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.935811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:46005 (local address 127.9.174.2:35337)
0807 08:38:15.936455 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:15.936473 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:15.936596 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:15.936890 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:15.936893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:15.936915 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:15.937079 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:15.937086 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.938130 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.938133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:15.939344 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:15.939357 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.940420 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.940425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:15.940512 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.940530 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 08:38:15.940598 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 08:38:15.940649 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":24,"thread_start_us":204,"threads_started":1}
I20260807 08:38:15.940882 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.936054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:35337 (local address 127.9.174.1:46005)
0807 08:38:15.936427 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:15.936434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:15.936584 (+ 150us) server_negotiation.cc:407] Connection header received
0807 08:38:15.936706 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:15.936709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:15.936758 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:15.936840 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:15.937207 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.937982 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.939484 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.940144 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.940251 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.940330 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:15.940658 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 08:38:15.940750 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":132,"threads_started":1}
I20260807 08:38:15.941154 1725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f805244abd14d8fb280947432f7ac95" candidate_uuid: "47fa071b785242bdb0861aa94d868819" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6016cae0ab49b187362a8a6cea5acf" is_pre_election: true
I20260807 08:38:15.941599 1725 raft_consensus.cc:2454] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47fa071b785242bdb0861aa94d868819 in term 0.
I20260807 08:38:15.941561 1322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f805244abd14d8fb280947432f7ac95" candidate_uuid: "47fa071b785242bdb0861aa94d868819" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "757fe0884d894fa380b34c2928462b9f" is_pre_election: true
I20260807 08:38:15.941936 1322 raft_consensus.cc:2454] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47fa071b785242bdb0861aa94d868819 in term 0.
I20260807 08:38:15.942288 1434 leader_election.cc:302] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47fa071b785242bdb0861aa94d868819, fc6016cae0ab49b187362a8a6cea5acf; no voters:
I20260807 08:38:15.942682 1822 raft_consensus.cc:2781] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:38:15.942781 1822 raft_consensus.cc:482] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:38:15.942895 1822 raft_consensus.cc:3037] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:15.944377 1822 raft_consensus.cc:504] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:15.944802 1822 leader_election.cc:288] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [CANDIDATE]: Term 1 election: Requested vote from peers fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549), 757fe0884d894fa380b34c2928462b9f (127.9.174.1:46005)
I20260807 08:38:15.945262 1322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f805244abd14d8fb280947432f7ac95" candidate_uuid: "47fa071b785242bdb0861aa94d868819" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "757fe0884d894fa380b34c2928462b9f"
I20260807 08:38:15.945262 1725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f805244abd14d8fb280947432f7ac95" candidate_uuid: "47fa071b785242bdb0861aa94d868819" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6016cae0ab49b187362a8a6cea5acf"
I20260807 08:38:15.945426 1322 raft_consensus.cc:3037] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:15.945426 1725 raft_consensus.cc:3037] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:15.947146 1725 raft_consensus.cc:2454] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47fa071b785242bdb0861aa94d868819 in term 1.
I20260807 08:38:15.947146 1322 raft_consensus.cc:2454] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47fa071b785242bdb0861aa94d868819 in term 1.
I20260807 08:38:15.947476 1434 leader_election.cc:302] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47fa071b785242bdb0861aa94d868819, 757fe0884d894fa380b34c2928462b9f; no voters:
I20260807 08:38:15.947685 1822 raft_consensus.cc:2781] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:38:15.947961 1822 raft_consensus.cc:686] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [term 1 LEADER]: Becoming Leader. State: Replica: 47fa071b785242bdb0861aa94d868819, State: Running, Role: LEADER
I20260807 08:38:15.948454 1822 consensus_queue.cc:237] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:15.952375 1083 catalog_manager.cc:5495] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47fa071b785242bdb0861aa94d868819 (127.9.174.2). New cstate: current_term: 1 leader_uuid: "47fa071b785242bdb0861aa94d868819" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } health_report { overall_health: UNKNOWN } } }
I20260807 08:38:15.994992 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:15.990915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46256 (local address 127.9.174.2:37327)
0807 08:38:15.991123 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:15.991128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:15.991147 (+ 19us) server_negotiation.cc:407] Connection header received
0807 08:38:15.991272 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:15.991275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:15.991344 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:15.991484 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:38:15.991756 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.992320 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.993226 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:15.993474 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:15.993591 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:15.993971 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:38:15.994575 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:38:15.994750 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:15.994781 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 08:38:15.994845 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 08:38:16.002925 1725 raft_consensus.cc:1261] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf [term 1 FOLLOWER]: Refusing update from remote peer 47fa071b785242bdb0861aa94d868819: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:38:16.002919 1322 raft_consensus.cc:1261] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Refusing update from remote peer 47fa071b785242bdb0861aa94d868819: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:38:16.003931 1826 consensus_queue.cc:1035] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:38:16.004192 1822 consensus_queue.cc:1035] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [LEADER]: Connected to new peer: Peer: permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:38:16.013474 1833 mvcc.cc:205] Tried to move back new op lower bound from 7315832406017556480 to 7315832405808205824. Current Snapshot: MvccSnapshot[applied={T|T < 7315832406017556480}]
I20260807 08:38:16.023222 1084 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:38:16.023476 1084 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:38:16.026544 1084 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Sending DeleteTablet for 3 replicas of tablet 1f805244abd14d8fb280947432f7ac95
I20260807 08:38:16.027539 1697 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f805244abd14d8fb280947432f7ac95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:38:16 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:47924
I20260807 08:38:16.027634 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 1149
I20260807 08:38:16.027840 1502 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f805244abd14d8fb280947432f7ac95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:38:16 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:46250
I20260807 08:38:16.031625 1844 tablet_replica.cc:330] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf: stopping tablet replica
I20260807 08:38:16.032140 1844 raft_consensus.cc:2229] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:38:16.032733 1844 raft_consensus.cc:2258] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:38:16.037227 1844 ts_tablet_manager.cc:1905] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:38:16.041728 1844 ts_tablet_manager.cc:1918] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:38:16.041963 1844 log.cc:1196] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/1f805244abd14d8fb280947432f7ac95
I20260807 08:38:16.042505 1844 ts_tablet_manager.cc:1939] T 1f805244abd14d8fb280947432f7ac95 P fc6016cae0ab49b187362a8a6cea5acf: Deleting consensus metadata
I20260807 08:38:16.044487 1064 catalog_manager.cc:4841] TS fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549): tablet 1f805244abd14d8fb280947432f7ac95 (table test-table [id=aaabf243f9a64feaaddb23c25c2d6d1c]) successfully deleted
W20260807 08:38:16.045861 1064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:46005 (error 108)
I20260807 08:38:16.046303 1845 tablet_replica.cc:330] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819: stopping tablet replica
I20260807 08:38:16.046746 1845 raft_consensus.cc:2229] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:38:16.047370 1845 raft_consensus.cc:2258] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 08:38:16.047838 1807 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:16.047057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:46005 (local address 127.0.0.1:55386)
0807 08:38:16.047219 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:16.047239 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:16.047356 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:16.047732 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:46005: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 08:38:16.048028 1064 catalog_manager.cc:4584] TS 757fe0884d894fa380b34c2928462b9f (127.9.174.1:46005): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1f805244abd14d8fb280947432f7ac95: Network error: Client connection negotiation failed: client connection to 127.9.174.1:46005: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:38:16.049108 1845 ts_tablet_manager.cc:1905] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:38:16.052024 1845 ts_tablet_manager.cc:1918] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:38:16.052160 1845 log.cc:1196] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/1f805244abd14d8fb280947432f7ac95
I20260807 08:38:16.052559 1845 ts_tablet_manager.cc:1939] T 1f805244abd14d8fb280947432f7ac95 P 47fa071b785242bdb0861aa94d868819: Deleting consensus metadata
I20260807 08:38:16.052839 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 1400
I20260807 08:38:16.074654 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 1599
I20260807 08:38:16.075102 1807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:16.074610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:37327 (local address 127.0.0.1:46258)
0807 08:38:16.074783 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:16.074901 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.2:37327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 08:38:16.094938 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 1032
I20260807 08:38:36.110535 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:43599
--webserver_interface=127.9.174.62
--webserver_port=44787
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:43599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:38:36.244249 1847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:36.244622 1847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:36.244669 1847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:36.249853 1847 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 08:38:36.249940 1847 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 08:38:36.249964 1847 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 08:38:36.249979 1847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:38:36.250000 1847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:36.250048 1847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:36.250083 1847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:38:36.250111 1847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:38:36.256549 1847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:43599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:43599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=44787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:36.258067 1847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:36.259454 1847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:38:36.271461 1847 server_base.cc:1034] running on GCE node
W20260807 08:38:36.271499 1858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:36.271473 1855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:36.271494 1856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:36.272632 1847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:36.274156 1847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:36.275484 1847 hybrid_clock.cc:648] HybridClock initialized: now 1786091916275438 us; error 68 us; skew 500 ppm
Aug 07 08:38:36 dist-test-slave-h3ft krb5kdc[32638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091916, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:36.283540 1847 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:38:36.284726 1847 webserver.cc:460] Webserver started at http://127.9.174.62:44787/ using document root <none> and password file <none>
I20260807 08:38:36.285422 1847 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:36.285499 1847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:36.290256 1847 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:38:36.292963 1870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:36.294466 1847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:38:36.294634 1847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "bbeaeef13c814174b459f3f7eedd66b2"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h3ft"
server_key: "fa11fb4127875411ce1036c3dbd720d0"
server_key_iv: "e263d3838e4a75220cebdafbca95a662"
server_key_version: "encryptionkey@0"
I20260807 08:38:36.295369 1847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:36.307947 1847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:36.312033 1847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:36.312285 1847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:36.321476 1847 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:43599
I20260807 08:38:36.321841 1953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:43599 every 8 connection(s)
I20260807 08:38:36.322854 1847 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:38:36.328907 1954 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:38:36.331233 1954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Bootstrap starting.
I20260807 08:38:36.331705 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 1847
I20260807 08:38:36.332386 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:46005
--local_ip_for_outbound_sockets=127.9.174.1
--tserver_master_addrs=127.9.174.62:43599
--webserver_port=40153
--webserver_interface=127.9.174.1
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:38:36.356817 1954 log.cc:824] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:36.371496 1954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:38:36.372071 1954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Bootstrap complete.
I20260807 08:38:36.377377 1954 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } }
I20260807 08:38:36.377913 1954 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbeaeef13c814174b459f3f7eedd66b2, State: Initialized, Role: FOLLOWER
I20260807 08:38:36.378530 1954 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } }
I20260807 08:38:36.378700 1954 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:38:36.378806 1954 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:38:36.378947 1954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 2 FOLLOWER]: Advancing to term 3
I20260807 08:38:36.381379 1954 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } }
I20260807 08:38:36.381837 1954 leader_election.cc:302] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbeaeef13c814174b459f3f7eedd66b2; no voters:
I20260807 08:38:36.382298 1954 leader_election.cc:288] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 08:38:36.382417 1958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 08:38:36.382668 1958 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [term 3 LEADER]: Becoming Leader. State: Replica: bbeaeef13c814174b459f3f7eedd66b2, State: Running, Role: LEADER
I20260807 08:38:36.383085 1958 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } }
I20260807 08:38:36.383904 1954 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:38:36.384831 1963 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbeaeef13c814174b459f3f7eedd66b2. Latest consensus state: current_term: 3 leader_uuid: "bbeaeef13c814174b459f3f7eedd66b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } } }
I20260807 08:38:36.384882 1961 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "bbeaeef13c814174b459f3f7eedd66b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeaeef13c814174b459f3f7eedd66b2" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43599 } } }
I20260807 08:38:36.385037 1961 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:36.384980 1963 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2 [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:36.387158 1971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:38:36.393002 1971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:38:36.393738 1971 catalog_manager.cc:1259] Loaded cluster ID: 2a5bb6cefc8141aa9b5f8e6a47fdb26b
I20260807 08:38:36.393838 1971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:38:36.396191 1971 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:38:36.396947 1971 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Loaded TSK: 0
I20260807 08:38:36.397058 1971 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bbeaeef13c814174b459f3f7eedd66b2: Loaded TSK: 1
F20260807 08:38:36.398393 1971 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc19bc279ad google::LogMessage::Fail() at ??:0
@ 0x7fc19bc2a88a google::LogMessage::SendToLog() at ??:0
@ 0x7fc19bc274de google::LogMessage::Flush() at ??:0
@ 0x7fc19bc28a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc19e8eb18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fc19e8e3f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fc1a658c7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fc1a656699c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fc1a6567178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fc1a65a8df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fc1a65c782b std::function<>::operator()() at ??:0
@ 0x7fc1a6566b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fc1a6567f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc1a6564579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fc1a65a8640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fc19de793d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc19de79c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fc19de7b3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fc19de6a512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc19f4ae6db start_thread at ??:0
@ 0x7fc19a72561f clone at ??:0
W20260807 08:38:36.509832 1956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:36.510306 1956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:36.510411 1956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:36.517239 1956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:36.517344 1956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:36.517477 1956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:38:36.526499 1956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:46005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=40153
--tserver_master_addrs=127.9.174.62:43599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:36.528405 1956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:36.530069 1956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:36.544665 2040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:36.544654 2036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:36.544854 2038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:36.545231 1956 server_base.cc:1034] running on GCE node
I20260807 08:38:36.545782 1956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:36.546720 1956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:36.548030 1956 hybrid_clock.cc:648] HybridClock initialized: now 1786091916547995 us; error 51 us; skew 500 ppm
Aug 07 08:38:36 dist-test-slave-h3ft krb5kdc[32638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091916, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:36.551560 1956 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:38:36.552682 1956 webserver.cc:460] Webserver started at http://127.9.174.1:40153/ using document root <none> and password file <none>
I20260807 08:38:36.553414 1956 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:36.553493 1956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:36.558655 1956 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:38:36.561651 2051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:36.563510 1956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 08:38:36.563678 1956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "757fe0884d894fa380b34c2928462b9f"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h3ft"
server_key: "43fdd10fa3b1658ba39cf0150d663ca7"
server_key_iv: "7d542f8083822e3778e5dc39b0eae33d"
server_key_version: "encryptionkey@0"
I20260807 08:38:36.564410 1956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:36.599921 1956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:36.604173 1956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:36.604446 1956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:36.605252 1956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:36.606698 2060 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:38:36.613554 1956 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:38:36.613704 1956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260807 08:38:36.613822 1956 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:38:36.616201 1956 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:38:36.616305 1956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260807 08:38:36.616633 2060 tablet_bootstrap.cc:492] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f: Bootstrap starting.
I20260807 08:38:36.630244 1956 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:46005
I20260807 08:38:36.630568 2242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:46005 every 8 connection(s)
I20260807 08:38:36.631716 1956 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:38:36.636957 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 1956
I20260807 08:38:36.637578 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:37327
--local_ip_for_outbound_sockets=127.9.174.2
--tserver_master_addrs=127.9.174.62:43599
--webserver_port=42187
--webserver_interface=127.9.174.2
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:38:36.644157 2060 log.cc:824] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:36.657011 2060 tablet_bootstrap.cc:492] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:38:36.657569 2060 tablet_bootstrap.cc:492] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f: Bootstrap complete.
I20260807 08:38:36.659549 2060 ts_tablet_manager.cc:1397] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f: Time spent bootstrapping tablet: real 0.043s user 0.023s sys 0.019s
I20260807 08:38:36.663622 2060 raft_consensus.cc:348] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:36.664244 2060 raft_consensus.cc:729] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 757fe0884d894fa380b34c2928462b9f, State: Initialized, Role: FOLLOWER
I20260807 08:38:36.664912 2060 consensus_queue.cc:260] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:36.666270 2060 ts_tablet_manager.cc:1428] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
W20260807 08:38:36.692180 2245 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:36.634418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:48687)
0807 08:38:36.635115 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:36.635182 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:36.636235 (+ 1053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:36.691210 (+ 54975us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":193,"threads_started":1}
W20260807 08:38:36.692854 2243 heartbeater.cc:644] Failed to heartbeat to 127.9.174.62:43599 (0 consecutive failures): Network error: Failed to ping master at 127.9.174.62:43599: Client connection negotiation failed: client connection to 127.9.174.62:43599: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:38:36.693928 2245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:36.693602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:58399)
0807 08:38:36.693776 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:36.693858 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 08:38:36.694783 2245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:36.694498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:49423)
0807 08:38:36.694651 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:36.694711 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 08:38:36.694994 2243 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:38:36.788954 2246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:36.789332 2246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:36.789382 2246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:36.793746 2246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:36.793836 2246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:36.793903 2246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:38:36.800036 2246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:37327
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=42187
--tserver_master_addrs=127.9.174.62:43599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:36.801509 2246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:36.802819 2246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:36.815126 2257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:36.815181 2258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:36.815146 2260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:36.815685 2246 server_base.cc:1034] running on GCE node
I20260807 08:38:36.816172 2246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:36.816968 2246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:36.818168 2246 hybrid_clock.cc:648] HybridClock initialized: now 1786091916818108 us; error 52 us; skew 500 ppm
Aug 07 08:38:36 dist-test-slave-h3ft krb5kdc[32638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091916, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:36.821465 2246 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:38:36.822561 2246 webserver.cc:460] Webserver started at http://127.9.174.2:42187/ using document root <none> and password file <none>
I20260807 08:38:36.823314 2246 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:36.823388 2246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:36.828094 2246 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:38:36.830756 2269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:36.832684 2246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:38:36.832867 2246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "47fa071b785242bdb0861aa94d868819"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h3ft"
server_key: "029cb0f1a7c134157aef0aa0919bddc3"
server_key_iv: "6b84a0d3876d478ed778d5b4bf95bff0"
server_key_version: "encryptionkey@0"
I20260807 08:38:36.833673 2246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:36.854905 2246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:36.859589 2246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:36.859866 2246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:36.860713 2246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:36.861887 2246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:36.861944 2246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:36.862037 2246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:36.862073 2246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:36.874958 2246 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:37327
I20260807 08:38:36.875044 2448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:37327 every 8 connection(s)
I20260807 08:38:36.876379 2246 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:38:36.879804 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 2246
I20260807 08:38:36.880394 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:35549
--local_ip_for_outbound_sockets=127.9.174.3
--tserver_master_addrs=127.9.174.62:43599
--webserver_port=33341
--webserver_interface=127.9.174.3
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:38:36.882128 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:36.879247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:43841)
0807 08:38:36.879959 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:36.880190 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":138,"threads_started":1}
W20260807 08:38:36.882670 2449 heartbeater.cc:644] Failed to heartbeat to 127.9.174.62:43599 (0 consecutive failures): Network error: Failed to ping master at 127.9.174.62:43599: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
I20260807 08:38:36.883584 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:36.883274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:59673)
0807 08:38:36.883427 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:36.883504 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 08:38:36.884464 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:36.884164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:60309)
0807 08:38:36.884301 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:36.884363 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 08:38:36.884702 2449 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:38:37.021358 2453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:37.021713 2453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:37.021763 2453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:37.026188 2453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:37.026271 2453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:37.026376 2453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:38:37.032253 2453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:35549
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=33341
--tserver_master_addrs=127.9.174.62:43599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:37.033829 2453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:37.035260 2453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:37.047498 2461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:37.047564 2460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:37.047581 2453 server_base.cc:1034] running on GCE node
W20260807 08:38:37.047508 2466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:37.048233 2453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:37.049060 2453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:37.050258 2453 hybrid_clock.cc:648] HybridClock initialized: now 1786091917050242 us; error 28 us; skew 500 ppm
Aug 07 08:38:37 dist-test-slave-h3ft krb5kdc[32638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091917, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:37.053755 2453 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:38:37.054848 2453 webserver.cc:460] Webserver started at http://127.9.174.3:33341/ using document root <none> and password file <none>
I20260807 08:38:37.055703 2453 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:37.055815 2453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:37.060611 2453 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:38:37.063292 2482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:37.064790 2453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:38:37.064975 2453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "fc6016cae0ab49b187362a8a6cea5acf"
format_stamp: "Formatted at 2026-08-07 08:37:54 on dist-test-slave-h3ft"
server_key: "6e9587a91159b439d72d5961a6b45b86"
server_key_iv: "654628d5a95d1c53eb95ed123e674640"
server_key_version: "encryptionkey@0"
I20260807 08:38:37.065667 2453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:37.106079 2453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:37.111059 2453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:37.111333 2453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:37.112208 2453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:37.113324 2453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:37.113401 2453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:37.113476 2453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:37.113528 2453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:37.124600 2453 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:35549
I20260807 08:38:37.124626 2665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:35549 every 8 connection(s)
I20260807 08:38:37.125946 2453 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:38:37.131160 2670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:37.128862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:39207)
0807 08:38:37.129465 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:37.129602 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":140,"threads_started":1}
W20260807 08:38:37.131685 2667 heartbeater.cc:644] Failed to heartbeat to 127.9.174.62:43599 (0 consecutive failures): Network error: Failed to ping master at 127.9.174.62:43599: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
I20260807 08:38:37.132465 2670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:37.132193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:45207)
0807 08:38:37.132328 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:37.132389 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 08:38:37.132822 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 2453
I20260807 08:38:37.133248 2670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:37.132989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:55485)
0807 08:38:37.133119 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:37.133178 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 08:38:37.133512 2667 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 08:38:37.696683 2673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:37.695944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:47963)
0807 08:38:37.696446 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:37.696529 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":147,"threads_started":1}
I20260807 08:38:37.886119 2674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:37.885525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:50429)
0807 08:38:37.885881 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:37.885953 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":97,"threads_started":1}
I20260807 08:38:38.004043 2675 raft_consensus.cc:482] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:38:38.004412 2675 raft_consensus.cc:504] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:38.005898 2675 leader_election.cc:288] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549), 47fa071b785242bdb0861aa94d868819 (127.9.174.2:37327)
Aug 07 08:38:38 dist-test-slave-h3ft krb5kdc[32638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091916, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.2@KRBTEST.COM
Aug 07 08:38:38 dist-test-slave-h3ft krb5kdc[32638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091916, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.3@KRBTEST.COM
I20260807 08:38:38.022395 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:38.006473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:50389 (local address 127.9.174.3:35549)
0807 08:38:38.006964 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:38.006986 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:38.007076 (+ 90us) server_negotiation.cc:407] Connection header received
0807 08:38:38.007974 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:38.007982 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:38.008416 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:38.008790 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:38.010318 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:38.011489 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:38.012867 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:38.013382 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:38.013595 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:38.017064 (+ 3469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:38.017081 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:38.017094 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:38.017138 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:38.019320 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:38.019881 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:38.019891 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:38.019898 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:38.020046 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:38.020526 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:38.020529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:38.020531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:38.021091 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:38.021203 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:38.021844 (+ 641us) server_negotiation.cc:299] Negotiation successful
0807 08:38:38.022143 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":164,"threads_started":1}
I20260807 08:38:38.022379 2678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:38.006585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:37327 (local address 127.9.174.1:50381)
0807 08:38:38.007087 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:38.007102 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:38.007212 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:38.009296 (+ 2084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:38.009300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:38.009560 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:38.010116 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:38.010136 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:38.011649 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:38.011657 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:38.012729 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:38.012741 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:38.013602 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:38.013606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:38.013715 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:38.014585 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:38.014598 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:38.016517 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:38.019851 (+ 3334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:38.019853 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:38.019855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:38.020126 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:38.020526 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:38.020529 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:38.020531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:38.020624 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:38.021325 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:38.021328 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:38.021511 (+ 183us) client_negotiation.cc:769] Sending connection context
0807 08:38:38.021726 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 08:38:38.022153 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":94,"spinlock_wait_cycles":31744,"thread_start_us":108,"threads_started":1}
I20260807 08:38:38.022395 2679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:38.006864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:50381 (local address 127.9.174.2:37327)
0807 08:38:38.007308 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:38.007331 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:38.007366 (+ 35us) server_negotiation.cc:407] Connection header received
0807 08:38:38.008072 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:38.008078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:38.008416 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:38.008791 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:38.010320 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:38.011466 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:38.012873 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:38.013468 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:38.013656 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:38.016765 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:38.016771 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:38.016784 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:38.016814 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:38.019320 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:38.020255 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:38.020261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:38.020266 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:38.020408 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:38.020727 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:38.020730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:38.020731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:38.021204 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:38.021339 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:38.021847 (+ 508us) server_negotiation.cc:299] Negotiation successful
0807 08:38:38.022168 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":80,"threads_started":1}
I20260807 08:38:38.022395 2673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:38.006250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:35549 (local address 127.9.174.1:50389)
0807 08:38:38.006412 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:38.006432 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:38.006600 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:38.008832 (+ 2232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:38.008840 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:38.009538 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:38.010119 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:38.010135 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:38.011645 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:38.011649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:38.012734 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:38.012745 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:38.013524 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:38.013528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:38.013706 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:38.014532 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:38.014550 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:38.016853 (+ 2303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:38.019456 (+ 2603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:38.019462 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:38.019474 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:38.019751 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:38.020177 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:38.020181 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:38.020183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:38.020400 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:38.021243 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:38.021253 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:38.021512 (+ 259us) client_negotiation.cc:769] Sending connection context
0807 08:38:38.021727 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 08:38:38.022248 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":442,"spinlock_wait_cycles":2688}
W20260807 08:38:38.025569 2057 leader_election.cc:341] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549): Not found: Tablet not found: 1f805244abd14d8fb280947432f7ac95
W20260807 08:38:38.025837 2057 leader_election.cc:341] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 47fa071b785242bdb0861aa94d868819 (127.9.174.2:37327): Not found: Tablet not found: 1f805244abd14d8fb280947432f7ac95
I20260807 08:38:38.025913 2057 leader_election.cc:302] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 757fe0884d894fa380b34c2928462b9f; no voters: 47fa071b785242bdb0861aa94d868819, fc6016cae0ab49b187362a8a6cea5acf
I20260807 08:38:38.026266 2675 raft_consensus.cc:2726] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:38:38.134756 2682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:38.134217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:55535)
0807 08:38:38.134556 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:38.134624 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":96,"threads_started":1}
I20260807 08:38:38.698516 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:38.697853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:54207)
0807 08:38:38.698287 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:38.698369 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":109,"threads_started":1}
I20260807 08:38:38.887943 2684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:38.887337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:41597)
0807 08:38:38.887722 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:38.887799 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":105,"threads_started":1}
I20260807 08:38:39.136379 2688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:39.135767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:56435)
0807 08:38:39.136149 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:39.136223 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":104,"threads_started":1}
I20260807 08:38:39.700523 2691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:39.699830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:36319)
0807 08:38:39.700272 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:39.700354 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":132,"threads_started":1}
I20260807 08:38:39.889643 2692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:39.889032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:48263)
0807 08:38:39.889412 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:39.889497 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":114,"threads_started":1}
I20260807 08:38:39.892856 2693 raft_consensus.cc:482] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:38:39.893067 2693 raft_consensus.cc:504] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:39.893574 2693 leader_election.cc:288] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549), 47fa071b785242bdb0861aa94d868819 (127.9.174.2:37327)
W20260807 08:38:39.894659 2057 leader_election.cc:341] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549): Not found: Tablet not found: 1f805244abd14d8fb280947432f7ac95
W20260807 08:38:39.894802 2057 leader_election.cc:341] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 47fa071b785242bdb0861aa94d868819 (127.9.174.2:37327): Not found: Tablet not found: 1f805244abd14d8fb280947432f7ac95
I20260807 08:38:39.894840 2057 leader_election.cc:302] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 757fe0884d894fa380b34c2928462b9f; no voters: 47fa071b785242bdb0861aa94d868819, fc6016cae0ab49b187362a8a6cea5acf
I20260807 08:38:39.895042 2693 raft_consensus.cc:2726] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:38:40.138000 2696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:40.137352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:47645)
0807 08:38:40.137751 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:40.137824 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":144,"threads_started":1}
I20260807 08:38:40.702256 2698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:40.701618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:54941)
0807 08:38:40.702016 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:40.702104 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":109,"threads_started":1}
I20260807 08:38:40.891464 2699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:40.890776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:59025)
0807 08:38:40.891208 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:40.891300 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":110,"threads_started":1}
I20260807 08:38:41.139840 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:41.139181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:60363)
0807 08:38:41.139544 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:41.139625 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":156,"threads_started":1}
I20260807 08:38:41.621771 2703 raft_consensus.cc:482] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:38:41.622007 2703 raft_consensus.cc:504] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:41.622695 2703 leader_election.cc:288] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549), 47fa071b785242bdb0861aa94d868819 (127.9.174.2:37327)
W20260807 08:38:41.623752 2057 leader_election.cc:341] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549): Not found: Tablet not found: 1f805244abd14d8fb280947432f7ac95
W20260807 08:38:41.623950 2057 leader_election.cc:341] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 47fa071b785242bdb0861aa94d868819 (127.9.174.2:37327): Not found: Tablet not found: 1f805244abd14d8fb280947432f7ac95
I20260807 08:38:41.624018 2057 leader_election.cc:302] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 757fe0884d894fa380b34c2928462b9f; no voters: 47fa071b785242bdb0861aa94d868819, fc6016cae0ab49b187362a8a6cea5acf
I20260807 08:38:41.624191 2703 raft_consensus.cc:2726] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:38:41.704524 2705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:41.703840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:39647)
0807 08:38:41.704266 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:41.704371 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":106,"threads_started":1}
I20260807 08:38:41.893378 2706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:41.892749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:44799)
0807 08:38:41.893153 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:41.893237 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":139,"threads_started":1}
I20260807 08:38:42.141769 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:42.141166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:44479)
0807 08:38:42.141547 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:42.141630 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":107,"threads_started":1}
I20260807 08:38:42.706581 2708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:42.705910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:43373)
0807 08:38:42.706335 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:42.706424 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":125,"threads_started":1}
I20260807 08:38:42.895146 2709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:42.894467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:33087)
0807 08:38:42.894850 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:42.894925 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":154,"threads_started":1}
I20260807 08:38:43.139655 2711 raft_consensus.cc:482] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:38:43.139856 2711 raft_consensus.cc:504] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:43.140522 2711 leader_election.cc:288] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549), 47fa071b785242bdb0861aa94d868819 (127.9.174.2:37327)
W20260807 08:38:43.141603 2057 leader_election.cc:341] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549): Not found: Tablet not found: 1f805244abd14d8fb280947432f7ac95
W20260807 08:38:43.141786 2057 leader_election.cc:341] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 47fa071b785242bdb0861aa94d868819 (127.9.174.2:37327): Not found: Tablet not found: 1f805244abd14d8fb280947432f7ac95
I20260807 08:38:43.141884 2057 leader_election.cc:302] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 757fe0884d894fa380b34c2928462b9f; no voters: 47fa071b785242bdb0861aa94d868819, fc6016cae0ab49b187362a8a6cea5acf
I20260807 08:38:43.142117 2711 raft_consensus.cc:2726] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:38:43.143216 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:43.142645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:55195)
0807 08:38:43.142971 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:43.143080 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":141,"threads_started":1}
I20260807 08:38:43.708500 2713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:43.707815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:54635)
0807 08:38:43.708258 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:43.708343 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":112,"threads_started":1}
I20260807 08:38:43.896776 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:43.896135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:60573)
0807 08:38:43.896558 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:43.896628 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":111,"threads_started":1}
I20260807 08:38:44.144793 2716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:44.144181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:38651)
0807 08:38:44.144581 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:44.144652 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":100,"threads_started":1}
I20260807 08:38:44.710361 2717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:44.709782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:43135)
0807 08:38:44.710145 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:44.710223 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":111,"threads_started":1}
I20260807 08:38:44.898435 2719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:44.897829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:49889)
0807 08:38:44.898203 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:44.898282 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":108,"threads_started":1}
I20260807 08:38:45.146384 2720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:45.145798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:38283)
0807 08:38:45.146161 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:45.146241 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":117,"threads_started":1}
I20260807 08:38:45.712440 2721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:45.711727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:41473)
0807 08:38:45.712160 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:45.712264 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":120,"threads_started":1}
I20260807 08:38:45.900249 2723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:45.899639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:54989)
0807 08:38:45.900008 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:45.900091 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":103,"threads_started":1}
I20260807 08:38:46.148043 2724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:46.147491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:40157)
0807 08:38:46.147832 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:46.147904 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":108,"threads_started":1}
I20260807 08:38:46.714555 2725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:46.713875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:59445)
0807 08:38:46.714281 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:46.714380 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":139,"threads_started":1}
I20260807 08:38:46.902086 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:46.901447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:49789)
0807 08:38:46.901842 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:46.901939 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":121,"threads_started":1}
I20260807 08:38:47.149780 2729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:47.149168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:38573)
0807 08:38:47.149555 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:47.149633 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":107,"threads_started":1}
I20260807 08:38:47.175369 2730 raft_consensus.cc:482] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:38:47.175563 2730 raft_consensus.cc:504] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6016cae0ab49b187362a8a6cea5acf" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35549 } } peers { permanent_uuid: "47fa071b785242bdb0861aa94d868819" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 37327 } } peers { permanent_uuid: "757fe0884d894fa380b34c2928462b9f" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 46005 } }
I20260807 08:38:47.176205 2730 leader_election.cc:288] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549), 47fa071b785242bdb0861aa94d868819 (127.9.174.2:37327)
W20260807 08:38:47.177353 2057 leader_election.cc:341] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc6016cae0ab49b187362a8a6cea5acf (127.9.174.3:35549): Not found: Tablet not found: 1f805244abd14d8fb280947432f7ac95
W20260807 08:38:47.177532 2057 leader_election.cc:341] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 47fa071b785242bdb0861aa94d868819 (127.9.174.2:37327): Not found: Tablet not found: 1f805244abd14d8fb280947432f7ac95
I20260807 08:38:47.177593 2057 leader_election.cc:302] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 757fe0884d894fa380b34c2928462b9f; no voters: 47fa071b785242bdb0861aa94d868819, fc6016cae0ab49b187362a8a6cea5acf
I20260807 08:38:47.177788 2730 raft_consensus.cc:2726] T 1f805244abd14d8fb280947432f7ac95 P 757fe0884d894fa380b34c2928462b9f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:38:47.716524 2734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:47.715843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:36121)
0807 08:38:47.716269 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:47.716369 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":149,"threads_started":1}
I20260807 08:38:47.903872 2739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:47.903278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:50563)
0807 08:38:47.903644 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:47.903726 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":110,"threads_started":1}
I20260807 08:38:48.151602 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:48.150913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:48873)
0807 08:38:48.151353 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:48.151434 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":171,"threads_started":1}
I20260807 08:38:48.718315 2741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:48.717682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:51713)
0807 08:38:48.718078 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:48.718170 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":155,"threads_started":1}
I20260807 08:38:48.905551 2742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:48.904972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:49991)
0807 08:38:48.905346 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:48.905414 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":146,"threads_started":1}
I20260807 08:38:49.153524 2743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:49.152731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:51033)
0807 08:38:49.153223 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:49.153316 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":167,"threads_started":1}
I20260807 08:38:49.719846 2744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:49.719298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:45693)
0807 08:38:49.719644 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:49.719708 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":110,"threads_started":1}
I20260807 08:38:49.907353 2745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:49.906702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:51147)
0807 08:38:49.907132 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:49.907208 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":141,"threads_started":1}
I20260807 08:38:50.155409 2746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:50.154626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:45197)
0807 08:38:50.155138 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:50.155213 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":155,"threads_started":1}
I20260807 08:38:50.721583 2747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:50.720978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:53739)
0807 08:38:50.721357 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:50.721431 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":141,"threads_started":1}
I20260807 08:38:50.908963 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:50.908351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:55833)
0807 08:38:50.908747 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:50.908823 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":158,"threads_started":1}
I20260807 08:38:51.157016 2750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:51.156413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:37369)
0807 08:38:51.156807 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:51.156878 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":145,"threads_started":1}
I20260807 08:38:51.723640 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:51.723044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.1:36707)
0807 08:38:51.723415 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:51.723496 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":116,"threads_started":1}
I20260807 08:38:51.910671 2752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:51.910110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.2:35507)
0807 08:38:51.910452 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:51.910538 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":128,"threads_started":1}
I20260807 08:38:52.142338 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 1956
I20260807 08:38:52.159071 2753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:52.158331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43599 (local address 127.9.174.3:52907)
0807 08:38:52.158735 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:52.158814 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.62:43599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":131,"threads_started":1}
I20260807 08:38:52.163074 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 2246
I20260807 08:38:52.180904 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 2453
2026-08-07T08:38:52Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (60575 ms)
[----------] 30 tests from SecurityITest (276436 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:38:52 dist-test-slave-h3ft krb5kdc[2757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:38:52 dist-test-slave-h3ft krb5kdc[2757](info): set up 2 sockets
Aug 07 08:38:52 dist-test-slave-h3ft krb5kdc[2757](info): commencing operation
krb5kdc: starting...
W20260807 08:38:53.475262 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:38:53 dist-test-slave-h3ft krb5kdc[2757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:38:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:38:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:38:53.630223 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:42137
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:44951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:42137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:38:53.767712 2779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:53.768239 2779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:53.768363 2779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:53.773226 2779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:38:53.773357 2779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:53.773394 2779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:53.773427 2779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:38:53.773487 2779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:38:53.779644 2779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:44951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:42137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:42137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:53.781243 2779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:53.782642 2779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:53.793771 2790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:53.793787 2794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:53.793845 2779 server_base.cc:1034] running on GCE node
W20260807 08:38:53.793918 2791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:53.794946 2779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:53.796484 2779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:53.797683 2779 hybrid_clock.cc:648] HybridClock initialized: now 1786091933797653 us; error 44 us; skew 500 ppm
Aug 07 08:38:53 dist-test-slave-h3ft krb5kdc[2757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091933, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:53.801249 2779 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:38:53.802431 2779 webserver.cc:460] Webserver started at http://127.9.174.62:38915/ using document root <none> and password file <none>
I20260807 08:38:53.803261 2779 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:53.803370 2779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:53.803669 2779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:53.806382 2779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "cb132b1772df48da8bfc77211708b42f"
format_stamp: "Formatted at 2026-08-07 08:38:53 on dist-test-slave-h3ft"
server_key: "11b8d0ce6004f34c04b74ab9e7fa5d57"
server_key_iv: "a395b1a1cc50f8e10eaa01961d630dac"
server_key_version: "encryptionkey@0"
I20260807 08:38:53.807267 2779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "cb132b1772df48da8bfc77211708b42f"
format_stamp: "Formatted at 2026-08-07 08:38:53 on dist-test-slave-h3ft"
server_key: "11b8d0ce6004f34c04b74ab9e7fa5d57"
server_key_iv: "a395b1a1cc50f8e10eaa01961d630dac"
server_key_version: "encryptionkey@0"
I20260807 08:38:53.812356 2779 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 08:38:53.815611 2806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:53.817297 2779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:38:53.817469 2779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "cb132b1772df48da8bfc77211708b42f"
format_stamp: "Formatted at 2026-08-07 08:38:53 on dist-test-slave-h3ft"
server_key: "11b8d0ce6004f34c04b74ab9e7fa5d57"
server_key_iv: "a395b1a1cc50f8e10eaa01961d630dac"
server_key_version: "encryptionkey@0"
I20260807 08:38:53.817646 2779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:53.831655 2779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:53.835932 2779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:53.836207 2779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:53.846148 2779 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:42137
I20260807 08:38:53.846158 2880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:42137 every 8 connection(s)
I20260807 08:38:53.847782 2779 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:38:53.850363 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 2779
I20260807 08:38:53.850519 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:38:53.851044 9912 external_mini_cluster.cc:1428] Setting key 3b92fae44a2ed9662e9d6093cdd0777d
I20260807 08:38:53.852492 2881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:53.861783 2881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f: Bootstrap starting.
Aug 07 08:38:53 dist-test-slave-h3ft krb5kdc[2757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:38:53.866267 2881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:53.867511 2881 log.cc:824] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:53.872097 2881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f: No bootstrap required, opened a new log
I20260807 08:38:53.873417 2885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:53.853385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37684 (local address 127.9.174.62:42137)
0807 08:38:53.853886 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:53.853908 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:53.853960 (+ 52us) server_negotiation.cc:407] Connection header received
0807 08:38:53.854944 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:53.854970 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:53.855495 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:53.855996 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:53.857290 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:53.858885 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:53.860368 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:53.861137 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:53.861326 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:53.865263 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:53.865272 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:53.865299 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:53.865356 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:53.868438 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:53.869245 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:53.869256 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:53.869268 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:53.869511 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:53.869879 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:53.869886 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:53.869890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:53.870629 (+ 739us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:53.870885 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:53.871769 (+ 884us) server_negotiation.cc:299] Negotiation successful
0807 08:38:53.872170 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":165,"threads_started":1}
I20260807 08:38:53.877338 2881 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb132b1772df48da8bfc77211708b42f" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42137 } }
I20260807 08:38:53.877612 2881 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:53.877693 2881 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb132b1772df48da8bfc77211708b42f, State: Initialized, Role: FOLLOWER
I20260807 08:38:53.878249 2881 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb132b1772df48da8bfc77211708b42f" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42137 } }
I20260807 08:38:53.878428 2881 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:38:53.878511 2881 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:38:53.878624 2881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:53.880407 2881 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb132b1772df48da8bfc77211708b42f" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42137 } }
I20260807 08:38:53.880854 2881 leader_election.cc:302] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb132b1772df48da8bfc77211708b42f; no voters:
I20260807 08:38:53.881256 2881 leader_election.cc:288] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:38:53.881520 2888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:38:53.881826 2888 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [term 1 LEADER]: Becoming Leader. State: Replica: cb132b1772df48da8bfc77211708b42f, State: Running, Role: LEADER
I20260807 08:38:53.882206 2888 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb132b1772df48da8bfc77211708b42f" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42137 } }
I20260807 08:38:53.883038 2881 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:38:53.884344 2888 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb132b1772df48da8bfc77211708b42f. Latest consensus state: current_term: 1 leader_uuid: "cb132b1772df48da8bfc77211708b42f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb132b1772df48da8bfc77211708b42f" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42137 } } }
I20260807 08:38:53.884486 2888 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:53.884855 2890 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb132b1772df48da8bfc77211708b42f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb132b1772df48da8bfc77211708b42f" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42137 } } }
I20260807 08:38:53.884956 2890 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:53.884955 2899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:38:53.889066 2899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:38:53.897624 2899 catalog_manager.cc:1347] Generated new cluster ID: 46a2a48a7ce44814bdf5ce60e2150763
I20260807 08:38:53.897720 2899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:38:53.912297 2899 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:38:53.913506 2899 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:38:53.919384 2899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f: Generated new TSK 0
I20260807 08:38:53.920225 2899 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:38:53.964790 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42137
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:44951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:38:54.122337 2918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:54.122730 2918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:54.122781 2918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:54.127321 2918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:54.127421 2918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:54.127491 2918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:38:54.133577 2918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:44951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:54.135200 2918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:54.136556 2918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:54.149288 2927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:54.149295 2924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:54.149288 2925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:54.149847 2918 server_base.cc:1034] running on GCE node
I20260807 08:38:54.150411 2918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:54.151338 2918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:54.152585 2918 hybrid_clock.cc:648] HybridClock initialized: now 1786091934152540 us; error 46 us; skew 500 ppm
Aug 07 08:38:54 dist-test-slave-h3ft krb5kdc[2757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091934, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:54.156222 2918 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:38:54.157423 2918 webserver.cc:460] Webserver started at http://127.9.174.1:46713/ using document root <none> and password file <none>
I20260807 08:38:54.158162 2918 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:54.158267 2918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:54.158563 2918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:54.161185 2918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "12a78dc0a883421985d3ab03c74e4096"
format_stamp: "Formatted at 2026-08-07 08:38:54 on dist-test-slave-h3ft"
server_key: "2f9a3a1deeb6ed4e0afa129665277160"
server_key_iv: "21d90099b3833b0f09b3571ba866d59f"
server_key_version: "encryptionkey@0"
I20260807 08:38:54.161893 2918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "12a78dc0a883421985d3ab03c74e4096"
format_stamp: "Formatted at 2026-08-07 08:38:54 on dist-test-slave-h3ft"
server_key: "2f9a3a1deeb6ed4e0afa129665277160"
server_key_iv: "21d90099b3833b0f09b3571ba866d59f"
server_key_version: "encryptionkey@0"
I20260807 08:38:54.166584 2918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260807 08:38:54.169641 2937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:54.171259 2918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 08:38:54.171442 2918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "12a78dc0a883421985d3ab03c74e4096"
format_stamp: "Formatted at 2026-08-07 08:38:54 on dist-test-slave-h3ft"
server_key: "2f9a3a1deeb6ed4e0afa129665277160"
server_key_iv: "21d90099b3833b0f09b3571ba866d59f"
server_key_version: "encryptionkey@0"
I20260807 08:38:54.171587 2918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:54.199922 2918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:54.204030 2918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:54.204312 2918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:54.205147 2918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:54.206452 2918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:54.206537 2918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:54.206626 2918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:54.206676 2918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:54.219751 2918 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:45383
I20260807 08:38:54.219776 3111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:45383 every 8 connection(s)
I20260807 08:38:54.220984 2918 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:38:54.227210 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 2918
I20260807 08:38:54.227401 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:38:54.227797 9912 external_mini_cluster.cc:1428] Setting key 05b01037c49cc76420d038bc4f0d5b4a
Aug 07 08:38:54 dist-test-slave-h3ft krb5kdc[2757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091934, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:38:54.239956 2885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.223677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:39729 (local address 127.9.174.62:42137)
0807 08:38:54.223877 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:54.223881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:54.224977 (+ 1096us) server_negotiation.cc:407] Connection header received
0807 08:38:54.226582 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:54.226586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:54.226650 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:54.226752 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:54.228856 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.229726 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.231689 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.232034 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.232159 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.235506 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:54.235510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:54.235512 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:54.235541 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:54.237374 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:54.237966 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:54.237969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:54.237971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:54.238075 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:54.238463 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:54.238467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:54.238469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:54.238698 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:54.238812 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:54.239566 (+ 754us) server_negotiation.cc:299] Negotiation successful
0807 08:38:54.239734 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 08:38:54.241024 3116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.224172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42137 (local address 127.9.174.1:39729)
0807 08:38:54.224759 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:54.224813 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:54.226311 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:54.227125 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:54.227137 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:54.227793 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:54.228628 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:54.228653 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.229891 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.229895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:54.231520 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:54.231537 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.232251 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.232259 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:54.232549 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.233496 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:54.233514 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:54.235287 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:54.237507 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:54.237514 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:54.237542 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:54.237843 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:54.238182 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:54.238185 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:54.238190 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:54.238350 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:54.238813 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:54.238821 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:54.239208 (+ 387us) client_negotiation.cc:769] Sending connection context
0807 08:38:54.239487 (+ 279us) client_negotiation.cc:240] Negotiation successful
0807 08:38:54.239903 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":158,"threads_started":1}
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
I20260807 08:38:54.242774 3114 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42137
I20260807 08:38:54.243189 3114 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:54.244004 3114 heartbeater.cc:511] Master 127.9.174.62:42137 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
I20260807 08:38:54.246358 2830 ts_manager.cc:194] Registered new tserver with Master: 12a78dc0a883421985d3ab03c74e4096 (127.9.174.1:45383)
I20260807 08:38:54.248353 2830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:39729
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:38:54.267316 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42137
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:44951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:38:54.405757 3122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:54.406132 3122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:54.406231 3122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:54.410708 3122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:54.410787 3122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:54.410853 3122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:38:54.416651 3122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:44951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:54.418083 3122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:54.419462 3122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:54.431473 3127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:54.431473 3132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:54.431529 3122 server_base.cc:1034] running on GCE node
W20260807 08:38:54.431460 3128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:54.432329 3122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:54.433310 3122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:54.434556 3122 hybrid_clock.cc:648] HybridClock initialized: now 1786091934434525 us; error 62 us; skew 500 ppm
Aug 07 08:38:54 dist-test-slave-h3ft krb5kdc[2757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091934, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:54.438313 3122 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:38:54.439404 3122 webserver.cc:460] Webserver started at http://127.9.174.2:45601/ using document root <none> and password file <none>
I20260807 08:38:54.440081 3122 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:54.440150 3122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:54.440357 3122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:54.442786 3122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "2b5914da10d24f6984f2b61ec50896ca"
format_stamp: "Formatted at 2026-08-07 08:38:54 on dist-test-slave-h3ft"
server_key: "2eb17e7a8005095240a0eeb267971ca7"
server_key_iv: "9bdff39c7c838a140bb1aba7a6ce3a14"
server_key_version: "encryptionkey@0"
I20260807 08:38:54.443490 3122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "2b5914da10d24f6984f2b61ec50896ca"
format_stamp: "Formatted at 2026-08-07 08:38:54 on dist-test-slave-h3ft"
server_key: "2eb17e7a8005095240a0eeb267971ca7"
server_key_iv: "9bdff39c7c838a140bb1aba7a6ce3a14"
server_key_version: "encryptionkey@0"
I20260807 08:38:54.447962 3122 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 08:38:54.450824 3141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:54.452423 3122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 08:38:54.452607 3122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "2b5914da10d24f6984f2b61ec50896ca"
format_stamp: "Formatted at 2026-08-07 08:38:54 on dist-test-slave-h3ft"
server_key: "2eb17e7a8005095240a0eeb267971ca7"
server_key_iv: "9bdff39c7c838a140bb1aba7a6ce3a14"
server_key_version: "encryptionkey@0"
I20260807 08:38:54.452759 3122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:54.477470 3122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:54.481709 3122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:54.481985 3122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:54.482869 3122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:54.484213 3122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:54.484287 3122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:54.484365 3122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:54.484417 3122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:54.496481 3122 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:43243
I20260807 08:38:54.496512 3324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:43243 every 8 connection(s)
I20260807 08:38:54.497790 3122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:38:54.507723 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 3122
I20260807 08:38:54.507895 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:38:54.508345 9912 external_mini_cluster.cc:1428] Setting key 049b5450aa2f23786a8ac4984dbd368d
Aug 07 08:38:54 dist-test-slave-h3ft krb5kdc[2757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091934, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:38:54.515272 2885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.500358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:49689 (local address 127.9.174.62:42137)
0807 08:38:54.500563 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:54.500568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:54.501533 (+ 965us) server_negotiation.cc:407] Connection header received
0807 08:38:54.503304 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:54.503309 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:54.503405 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:54.503511 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:54.504945 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.505551 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.506956 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.507271 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.507403 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.510837 (+ 3434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:54.510844 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:54.510850 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:54.510883 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:54.512790 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:54.513380 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:54.513384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:54.513386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:54.513486 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:54.513880 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:54.513884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:54.513886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:54.514118 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:54.514236 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:54.514862 (+ 626us) server_negotiation.cc:299] Negotiation successful
0807 08:38:54.515066 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 08:38:54.516470 3330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.500779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42137 (local address 127.9.174.2:49689)
0807 08:38:54.501350 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:54.501404 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:54.502989 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:54.503715 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:54.503722 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:54.504173 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:54.504772 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:54.504787 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.505687 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.505693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:54.506814 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:54.506825 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.507460 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.507464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:54.507671 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.508519 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:54.508539 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:54.510645 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:54.512923 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:54.512929 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:54.512944 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:54.513251 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:54.513599 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:54.513602 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:54.513604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:54.513773 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:54.514250 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:54.514259 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:54.514594 (+ 335us) client_negotiation.cc:769] Sending connection context
0807 08:38:54.514820 (+ 226us) client_negotiation.cc:240] Negotiation successful
0807 08:38:54.515258 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":156,"threads_started":1}
I20260807 08:38:54.517949 3326 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42137
I20260807 08:38:54.518301 3326 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:54.519186 3326 heartbeater.cc:511] Master 127.9.174.62:42137 requested a full tablet report, sending...
I20260807 08:38:54.520681 2830 ts_manager.cc:194] Registered new tserver with Master: 2b5914da10d24f6984f2b61ec50896ca (127.9.174.2:43243)
I20260807 08:38:54.521394 2830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:49689
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:38:54.544430 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42137
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:44951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:38:54.686657 3335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:54.687034 3335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:54.687086 3335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:54.692021 3335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:54.692101 3335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:54.692214 3335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:38:54.698299 3335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:44951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:54.699879 3335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:54.701231 3335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:54.713318 3349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:54.713382 3343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:54.713366 3344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:54.713812 3335 server_base.cc:1034] running on GCE node
I20260807 08:38:54.714361 3335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:54.715287 3335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:54.716470 3335 hybrid_clock.cc:648] HybridClock initialized: now 1786091934716434 us; error 47 us; skew 500 ppm
Aug 07 08:38:54 dist-test-slave-h3ft krb5kdc[2757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091934, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:54.723939 3335 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:38:54.725201 3335 webserver.cc:460] Webserver started at http://127.9.174.3:33909/ using document root <none> and password file <none>
I20260807 08:38:54.725946 3335 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:54.726023 3335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:54.726249 3335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:54.729000 3335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "044c79054e9c4acc9f6b8e38d7871f55"
format_stamp: "Formatted at 2026-08-07 08:38:54 on dist-test-slave-h3ft"
server_key: "7ce76c06bf07b3a82f1f6a6f7506e977"
server_key_iv: "69683431e12a68eae71b1d659aacfce4"
server_key_version: "encryptionkey@0"
I20260807 08:38:54.729681 3335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "044c79054e9c4acc9f6b8e38d7871f55"
format_stamp: "Formatted at 2026-08-07 08:38:54 on dist-test-slave-h3ft"
server_key: "7ce76c06bf07b3a82f1f6a6f7506e977"
server_key_iv: "69683431e12a68eae71b1d659aacfce4"
server_key_version: "encryptionkey@0"
I20260807 08:38:54.734584 3335 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:38:54.737957 3359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:54.739653 3335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:38:54.739828 3335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "044c79054e9c4acc9f6b8e38d7871f55"
format_stamp: "Formatted at 2026-08-07 08:38:54 on dist-test-slave-h3ft"
server_key: "7ce76c06bf07b3a82f1f6a6f7506e977"
server_key_iv: "69683431e12a68eae71b1d659aacfce4"
server_key_version: "encryptionkey@0"
I20260807 08:38:54.739985 3335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:54.756083 3335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:54.760517 3335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:54.760792 3335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:54.761636 3335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:54.762729 3335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:54.762810 3335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:54.762919 3335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:54.762964 3335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:54.774606 3335 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:42437
I20260807 08:38:54.774629 3535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:42437 every 8 connection(s)
I20260807 08:38:54.775925 3335 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:38:54.784643 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 3335
I20260807 08:38:54.784833 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:38:54.785207 9912 external_mini_cluster.cc:1428] Setting key 56cd462c952d9982053540455f2cc35d
Aug 07 08:38:54 dist-test-slave-h3ft krb5kdc[2757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091934, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:38:54.792923 2885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.778420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:42041 (local address 127.9.174.62:42137)
0807 08:38:54.778615 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:54.778619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:54.779642 (+ 1023us) server_negotiation.cc:407] Connection header received
0807 08:38:54.781285 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:54.781289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:54.781376 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:54.781496 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:54.782973 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.783708 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.785331 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.785602 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.785735 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.788917 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:54.788922 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:54.788925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:54.788958 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:54.790388 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:54.790972 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:54.790977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:54.790979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:54.791144 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:54.791542 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:54.791546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:54.791548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:54.791786 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:54.791919 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:54.792527 (+ 608us) server_negotiation.cc:299] Negotiation successful
0807 08:38:54.792714 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 08:38:54.794008 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.778851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42137 (local address 127.9.174.3:42041)
0807 08:38:54.779453 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:54.779505 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:54.781010 (+ 1505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:54.781715 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:54.781723 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:54.782171 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:54.782788 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:54.782806 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.783863 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.783866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:54.785179 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:54.785192 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.785797 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.785802 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:54.786023 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.786784 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:54.786803 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:54.788712 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:54.790523 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:54.790530 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:54.790550 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:54.790854 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:54.791250 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:54.791254 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:54.791256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:54.791433 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:54.791909 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:54.791917 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:54.792244 (+ 327us) client_negotiation.cc:769] Sending connection context
0807 08:38:54.792482 (+ 238us) client_negotiation.cc:240] Negotiation successful
0807 08:38:54.792918 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":190,"threads_started":1}
I20260807 08:38:54.795471 3536 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42137
I20260807 08:38:54.795799 3536 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:54.796512 3536 heartbeater.cc:511] Master 127.9.174.62:42137 requested a full tablet report, sending...
I20260807 08:38:54.798043 2830 ts_manager.cc:194] Registered new tserver with Master: 044c79054e9c4acc9f6b8e38d7871f55 (127.9.174.3:42437)
I20260807 08:38:54.798846 2830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:42041
I20260807 08:38:54.800127 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:38:54.811805 2885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.802636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37696 (local address 127.9.174.62:42137)
0807 08:38:54.802794 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:54.802798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:54.803064 (+ 266us) server_negotiation.cc:407] Connection header received
0807 08:38:54.803206 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:54.803208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:54.803258 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:54.803334 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:54.804197 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.804775 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.805659 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.805946 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.806036 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.807844 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:54.807848 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:54.807850 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:54.807919 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:54.809637 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:54.810224 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:54.810228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:54.810234 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:54.810346 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:54.810648 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:54.810650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:54.810652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:54.811090 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:54.811230 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:54.811502 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 08:38:54.811645 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 08:38:54.815750 2830 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:38:54.820250 2830 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:38:54.839648 3550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.832230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:45383 (local address 127.0.0.1:59978)
0807 08:38:54.833411 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:54.833484 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:54.833671 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:54.834380 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:54.834393 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:54.834423 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:54.834632 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:54.834640 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.836350 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.836357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:54.837625 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:54.837637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.838875 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.838879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:54.839052 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.839119 (+ 67us) client_negotiation.cc:769] Sending connection context
0807 08:38:54.839227 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 08:38:54.839418 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":28,"thread_start_us":140,"threads_started":1}
I20260807 08:38:54.839695 3557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.833190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:42437 (local address 127.0.0.1:34406)
0807 08:38:54.834346 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:54.834359 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:54.834444 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:54.835277 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:54.835280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:54.835299 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:54.835406 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:54.835412 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.836800 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.836804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:54.838063 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:54.838075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.839272 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.839276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:54.839372 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.839396 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 08:38:54.839456 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 08:38:54.839550 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1074,"mutex_wait_us":54,"thread_start_us":113,"threads_started":1}
I20260807 08:38:54.840029 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.832974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:43243 (local address 127.0.0.1:49800)
0807 08:38:54.834039 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:54.834054 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:54.834173 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:54.834652 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:54.834655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:54.834677 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:54.834813 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:54.834824 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.836453 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.836456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:54.837976 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:54.837988 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.839456 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.839474 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:54.839596 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.839620 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 08:38:54.839676 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 08:38:54.839765 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"mutex_wait_us":143,"thread_start_us":162,"threads_started":1}
I20260807 08:38:54.840525 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.833568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34406 (local address 127.9.174.3:42437)
0807 08:38:54.834056 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:54.834071 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:54.834541 (+ 470us) server_negotiation.cc:407] Connection header received
0807 08:38:54.834645 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:54.834650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:54.834855 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:54.835041 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:54.835546 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.836600 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.838214 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.839137 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.839261 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.839457 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:54.839518 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 08:38:54.839630 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":129,"threads_started":1}
I20260807 08:38:54.840865 3558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.833243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49800 (local address 127.9.174.2:43243)
0807 08:38:54.833919 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:54.833934 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:54.834174 (+ 240us) server_negotiation.cc:407] Connection header received
0807 08:38:54.834281 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:54.834290 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:54.834498 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:54.834988 (+ 490us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:54.835257 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.836241 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.838144 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.839311 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.839859 (+ 548us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.840508 (+ 649us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:54.840568 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 08:38:54.840680 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":216,"threads_started":1}
I20260807 08:38:54.840888 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.832408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59978 (local address 127.9.174.1:45383)
0807 08:38:54.832827 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:54.832839 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:54.833635 (+ 796us) server_negotiation.cc:407] Connection header received
0807 08:38:54.833891 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:54.833899 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:54.834176 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:54.834376 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:54.834785 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.836193 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.837780 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.838709 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.840404 (+ 1695us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.840562 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:54.840614 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 08:38:54.840714 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":109,"threads_started":1}
I20260807 08:38:54.844604 3222 tablet_service.cc:1467] Processing CreateTablet for tablet c9413670e75d4249bfe5ac6cdb384b30 (DEFAULT_TABLE table=test-table [id=f473ab98be6f4207bb88be3935f455f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:38:54.844594 3010 tablet_service.cc:1467] Processing CreateTablet for tablet c9413670e75d4249bfe5ac6cdb384b30 (DEFAULT_TABLE table=test-table [id=f473ab98be6f4207bb88be3935f455f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:38:54.846199 3222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9413670e75d4249bfe5ac6cdb384b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:54.844597 3436 tablet_service.cc:1467] Processing CreateTablet for tablet c9413670e75d4249bfe5ac6cdb384b30 (DEFAULT_TABLE table=test-table [id=f473ab98be6f4207bb88be3935f455f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:38:54.847733 3436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9413670e75d4249bfe5ac6cdb384b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:54.851464 3010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9413670e75d4249bfe5ac6cdb384b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:54.859135 3564 tablet_bootstrap.cc:492] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca: Bootstrap starting.
I20260807 08:38:54.860567 3567 tablet_bootstrap.cc:492] T c9413670e75d4249bfe5ac6cdb384b30 P 12a78dc0a883421985d3ab03c74e4096: Bootstrap starting.
I20260807 08:38:54.861663 3566 tablet_bootstrap.cc:492] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55: Bootstrap starting.
I20260807 08:38:54.862958 3564 tablet_bootstrap.cc:654] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:54.863324 3567 tablet_bootstrap.cc:654] T c9413670e75d4249bfe5ac6cdb384b30 P 12a78dc0a883421985d3ab03c74e4096: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:54.864104 3566 tablet_bootstrap.cc:654] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:54.864502 3564 log.cc:824] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:54.865691 3566 log.cc:824] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:54.865711 3567 log.cc:824] T c9413670e75d4249bfe5ac6cdb384b30 P 12a78dc0a883421985d3ab03c74e4096: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:54.867781 3564 tablet_bootstrap.cc:492] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca: No bootstrap required, opened a new log
I20260807 08:38:54.868108 3564 ts_tablet_manager.cc:1397] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 08:38:54.868281 3567 tablet_bootstrap.cc:492] T c9413670e75d4249bfe5ac6cdb384b30 P 12a78dc0a883421985d3ab03c74e4096: No bootstrap required, opened a new log
I20260807 08:38:54.868281 3566 tablet_bootstrap.cc:492] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55: No bootstrap required, opened a new log
I20260807 08:38:54.868498 3566 ts_tablet_manager.cc:1397] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 08:38:54.868498 3567 ts_tablet_manager.cc:1397] T c9413670e75d4249bfe5ac6cdb384b30 P 12a78dc0a883421985d3ab03c74e4096: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260807 08:38:54.872066 3564 raft_consensus.cc:348] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a78dc0a883421985d3ab03c74e4096" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 45383 } } peers { permanent_uuid: "2b5914da10d24f6984f2b61ec50896ca" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43243 } } peers { permanent_uuid: "044c79054e9c4acc9f6b8e38d7871f55" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 42437 } }
I20260807 08:38:54.872453 3564 raft_consensus.cc:374] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:54.872536 3564 raft_consensus.cc:729] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b5914da10d24f6984f2b61ec50896ca, State: Initialized, Role: FOLLOWER
I20260807 08:38:54.872978 3566 raft_consensus.cc:348] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a78dc0a883421985d3ab03c74e4096" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 45383 } } peers { permanent_uuid: "2b5914da10d24f6984f2b61ec50896ca" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43243 } } peers { permanent_uuid: "044c79054e9c4acc9f6b8e38d7871f55" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 42437 } }
I20260807 08:38:54.873308 3566 raft_consensus.cc:374] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:54.873366 3566 raft_consensus.cc:729] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 044c79054e9c4acc9f6b8e38d7871f55, State: Initialized, Role: FOLLOWER
I20260807 08:38:54.873265 3564 consensus_queue.cc:260] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a78dc0a883421985d3ab03c74e4096" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 45383 } } peers { permanent_uuid: "2b5914da10d24f6984f2b61ec50896ca" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43243 } } peers { permanent_uuid: "044c79054e9c4acc9f6b8e38d7871f55" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 42437 } }
I20260807 08:38:54.873800 3566 consensus_queue.cc:260] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a78dc0a883421985d3ab03c74e4096" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 45383 } } peers { permanent_uuid: "2b5914da10d24f6984f2b61ec50896ca" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43243 } } peers { permanent_uuid: "044c79054e9c4acc9f6b8e38d7871f55" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 42437 } }
I20260807 08:38:54.873719 3567 raft_consensus.cc:348] T c9413670e75d4249bfe5ac6cdb384b30 P 12a78dc0a883421985d3ab03c74e4096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a78dc0a883421985d3ab03c74e4096" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 45383 } } peers { permanent_uuid: "2b5914da10d24f6984f2b61ec50896ca" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43243 } } peers { permanent_uuid: "044c79054e9c4acc9f6b8e38d7871f55" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 42437 } }
I20260807 08:38:54.873976 3567 raft_consensus.cc:374] T c9413670e75d4249bfe5ac6cdb384b30 P 12a78dc0a883421985d3ab03c74e4096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:54.874027 3567 raft_consensus.cc:729] T c9413670e75d4249bfe5ac6cdb384b30 P 12a78dc0a883421985d3ab03c74e4096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12a78dc0a883421985d3ab03c74e4096, State: Initialized, Role: FOLLOWER
I20260807 08:38:54.874470 3567 consensus_queue.cc:260] T c9413670e75d4249bfe5ac6cdb384b30 P 12a78dc0a883421985d3ab03c74e4096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a78dc0a883421985d3ab03c74e4096" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 45383 } } peers { permanent_uuid: "2b5914da10d24f6984f2b61ec50896ca" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43243 } } peers { permanent_uuid: "044c79054e9c4acc9f6b8e38d7871f55" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 42437 } }
I20260807 08:38:54.874815 3564 ts_tablet_manager.cc:1428] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 08:38:54.874917 3566 ts_tablet_manager.cc:1428] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 08:38:54.875286 3326 heartbeater.cc:503] Master 127.9.174.62:42137 was elected leader, sending a full tablet report...
I20260807 08:38:54.875422 3536 heartbeater.cc:503] Master 127.9.174.62:42137 was elected leader, sending a full tablet report...
I20260807 08:38:54.875476 3567 ts_tablet_manager.cc:1428] T c9413670e75d4249bfe5ac6cdb384b30 P 12a78dc0a883421985d3ab03c74e4096: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260807 08:38:54.875857 3114 heartbeater.cc:503] Master 127.9.174.62:42137 was elected leader, sending a full tablet report...
I20260807 08:38:54.916664 3572 raft_consensus.cc:482] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:38:54.916991 3572 raft_consensus.cc:504] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a78dc0a883421985d3ab03c74e4096" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 45383 } } peers { permanent_uuid: "2b5914da10d24f6984f2b61ec50896ca" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43243 } } peers { permanent_uuid: "044c79054e9c4acc9f6b8e38d7871f55" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 42437 } }
I20260807 08:38:54.918579 3572 leader_election.cc:288] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12a78dc0a883421985d3ab03c74e4096 (127.9.174.1:45383), 044c79054e9c4acc9f6b8e38d7871f55 (127.9.174.3:42437)
I20260807 08:38:54.923966 3330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.918945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:45383 (local address 127.9.174.2:53591)
0807 08:38:54.919182 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:54.919220 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:54.919356 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:54.919652 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:54.919655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:54.919675 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:54.919842 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:54.919848 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.921025 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.921030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:54.922421 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:54.922433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.923407 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.923411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:54.923536 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.923566 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 08:38:54.923669 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 08:38:54.923772 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260807 08:38:54.923966 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.919145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:53591 (local address 127.9.174.1:45383)
0807 08:38:54.919355 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:54.919361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:54.919382 (+ 21us) server_negotiation.cc:407] Connection header received
0807 08:38:54.919461 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:54.919464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:54.919527 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:54.919622 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:54.919974 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.920786 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.922607 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.923264 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.923394 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.923491 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:54.923694 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 08:38:54.923794 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 08:38:54.924929 3576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.919387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:42437 (local address 127.9.174.2:60285)
0807 08:38:54.920190 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:54.920207 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:54.920335 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:54.920604 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:54.920607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:54.920624 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:54.920764 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:54.920770 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.922053 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.922056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:54.923163 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:54.923174 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.924440 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.924446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:54.924564 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.924590 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 08:38:54.924667 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 08:38:54.924749 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":165,"threads_started":1}
I20260807 08:38:54.924955 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.919605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:60285 (local address 127.9.174.3:42437)
0807 08:38:54.919774 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:54.919778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:54.920345 (+ 567us) server_negotiation.cc:407] Connection header received
0807 08:38:54.920427 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:54.920429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:54.920483 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:54.920571 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:54.921026 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.921888 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.923520 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.924104 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.924256 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.924344 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:54.924712 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 08:38:54.924816 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 08:38:54.925040 3048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9413670e75d4249bfe5ac6cdb384b30" candidate_uuid: "2b5914da10d24f6984f2b61ec50896ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12a78dc0a883421985d3ab03c74e4096" is_pre_election: true
I20260807 08:38:54.925518 3048 raft_consensus.cc:2454] T c9413670e75d4249bfe5ac6cdb384b30 P 12a78dc0a883421985d3ab03c74e4096 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b5914da10d24f6984f2b61ec50896ca in term 0.
I20260807 08:38:54.926297 3146 leader_election.cc:302] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12a78dc0a883421985d3ab03c74e4096, 2b5914da10d24f6984f2b61ec50896ca; no voters:
I20260807 08:38:54.926723 3572 raft_consensus.cc:2781] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:38:54.926708 3463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9413670e75d4249bfe5ac6cdb384b30" candidate_uuid: "2b5914da10d24f6984f2b61ec50896ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "044c79054e9c4acc9f6b8e38d7871f55" is_pre_election: true
I20260807 08:38:54.926849 3572 raft_consensus.cc:482] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:38:54.926899 3572 raft_consensus.cc:3037] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:54.927148 3463 raft_consensus.cc:2454] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b5914da10d24f6984f2b61ec50896ca in term 0.
I20260807 08:38:54.928488 3572 raft_consensus.cc:504] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a78dc0a883421985d3ab03c74e4096" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 45383 } } peers { permanent_uuid: "2b5914da10d24f6984f2b61ec50896ca" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43243 } } peers { permanent_uuid: "044c79054e9c4acc9f6b8e38d7871f55" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 42437 } }
I20260807 08:38:54.928874 3572 leader_election.cc:288] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [CANDIDATE]: Term 1 election: Requested vote from peers 12a78dc0a883421985d3ab03c74e4096 (127.9.174.1:45383), 044c79054e9c4acc9f6b8e38d7871f55 (127.9.174.3:42437)
I20260807 08:38:54.929488 3463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9413670e75d4249bfe5ac6cdb384b30" candidate_uuid: "2b5914da10d24f6984f2b61ec50896ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "044c79054e9c4acc9f6b8e38d7871f55"
I20260807 08:38:54.929489 3048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9413670e75d4249bfe5ac6cdb384b30" candidate_uuid: "2b5914da10d24f6984f2b61ec50896ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12a78dc0a883421985d3ab03c74e4096"
I20260807 08:38:54.929687 3048 raft_consensus.cc:3037] T c9413670e75d4249bfe5ac6cdb384b30 P 12a78dc0a883421985d3ab03c74e4096 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:54.929668 3463 raft_consensus.cc:3037] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:54.931176 3463 raft_consensus.cc:2454] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b5914da10d24f6984f2b61ec50896ca in term 1.
I20260807 08:38:54.931180 3048 raft_consensus.cc:2454] T c9413670e75d4249bfe5ac6cdb384b30 P 12a78dc0a883421985d3ab03c74e4096 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b5914da10d24f6984f2b61ec50896ca in term 1.
I20260807 08:38:54.931612 3146 leader_election.cc:302] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 044c79054e9c4acc9f6b8e38d7871f55, 2b5914da10d24f6984f2b61ec50896ca; no voters:
I20260807 08:38:54.931838 3572 raft_consensus.cc:2781] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:38:54.932159 3572 raft_consensus.cc:686] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [term 1 LEADER]: Becoming Leader. State: Replica: 2b5914da10d24f6984f2b61ec50896ca, State: Running, Role: LEADER
I20260807 08:38:54.932641 3572 consensus_queue.cc:237] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a78dc0a883421985d3ab03c74e4096" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 45383 } } peers { permanent_uuid: "2b5914da10d24f6984f2b61ec50896ca" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43243 } } peers { permanent_uuid: "044c79054e9c4acc9f6b8e38d7871f55" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 42437 } }
I20260807 08:38:54.937093 2828 catalog_manager.cc:5495] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b5914da10d24f6984f2b61ec50896ca (127.9.174.2). New cstate: current_term: 1 leader_uuid: "2b5914da10d24f6984f2b61ec50896ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a78dc0a883421985d3ab03c74e4096" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 45383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b5914da10d24f6984f2b61ec50896ca" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 43243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "044c79054e9c4acc9f6b8e38d7871f55" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 42437 } health_report { overall_health: UNKNOWN } } }
I20260807 08:38:54.972252 3558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:54.967112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49802 (local address 127.9.174.2:43243)
0807 08:38:54.967282 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:54.967286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:54.967301 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:38:54.967414 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:54.967416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:54.967498 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:54.967623 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:38:54.967969 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.968551 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.970012 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:54.970331 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:54.970448 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:54.970910 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:38:54.971664 (+ 754us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:38:54.971785 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:54.971997 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 08:38:54.972086 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 08:38:54.972968 3115 tablet.cc:2369] T c9413670e75d4249bfe5ac6cdb384b30 P 12a78dc0a883421985d3ab03c74e4096: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:38:54.980595 3048 raft_consensus.cc:1261] T c9413670e75d4249bfe5ac6cdb384b30 P 12a78dc0a883421985d3ab03c74e4096 [term 1 FOLLOWER]: Refusing update from remote peer 2b5914da10d24f6984f2b61ec50896ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:38:54.980595 3463 raft_consensus.cc:1261] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55 [term 1 FOLLOWER]: Refusing update from remote peer 2b5914da10d24f6984f2b61ec50896ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:38:54.981542 3572 consensus_queue.cc:1035] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12a78dc0a883421985d3ab03c74e4096" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 45383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:38:54.981879 3577 consensus_queue.cc:1035] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "044c79054e9c4acc9f6b8e38d7871f55" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 42437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:38:54.991307 3581 mvcc.cc:205] Tried to move back new op lower bound from 7315832565670486016 to 7315832565487501312. Current Snapshot: MvccSnapshot[applied={T|T < 7315832565670486016}]
I20260807 08:38:54.993580 3582 mvcc.cc:205] Tried to move back new op lower bound from 7315832565670486016 to 7315832565487501312. Current Snapshot: MvccSnapshot[applied={T|T < 7315832565670486016}]
I20260807 08:38:55.005496 2830 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:38:55.005818 2830 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:38:55.009824 2830 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb132b1772df48da8bfc77211708b42f: Sending DeleteTablet for 3 replicas of tablet c9413670e75d4249bfe5ac6cdb384b30
I20260807 08:38:55.011313 3010 tablet_service.cc:1514] Processing DeleteTablet for tablet c9413670e75d4249bfe5ac6cdb384b30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:38:55 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:59978
I20260807 08:38:55.011370 3222 tablet_service.cc:1514] Processing DeleteTablet for tablet c9413670e75d4249bfe5ac6cdb384b30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:38:55 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:49800
I20260807 08:38:55.012355 3598 tablet_replica.cc:330] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca: stopping tablet replica
I20260807 08:38:55.012430 3436 tablet_service.cc:1514] Processing DeleteTablet for tablet c9413670e75d4249bfe5ac6cdb384b30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:38:55 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:34406
I20260807 08:38:55.012923 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 2918
I20260807 08:38:55.012920 3598 raft_consensus.cc:2229] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:38:55.016733 3598 raft_consensus.cc:2258] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:38:55.019913 3599 tablet_replica.cc:330] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55: stopping tablet replica
I20260807 08:38:55.020452 3599 raft_consensus.cc:2229] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:38:55.020942 3599 raft_consensus.cc:2258] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:38:55.022719 3598 ts_tablet_manager.cc:1905] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:38:55.027415 3598 ts_tablet_manager.cc:1918] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:38:55.027565 3598 log.cc:1196] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/c9413670e75d4249bfe5ac6cdb384b30
I20260807 08:38:55.027992 3598 ts_tablet_manager.cc:1939] T c9413670e75d4249bfe5ac6cdb384b30 P 2b5914da10d24f6984f2b61ec50896ca: Deleting consensus metadata
I20260807 08:38:55.028465 3599 ts_tablet_manager.cc:1905] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:38:55.029949 2807 catalog_manager.cc:4841] TS 2b5914da10d24f6984f2b61ec50896ca (127.9.174.2:43243): tablet c9413670e75d4249bfe5ac6cdb384b30 (table test-table [id=f473ab98be6f4207bb88be3935f455f3]) successfully deleted
I20260807 08:38:55.031971 3599 ts_tablet_manager.cc:1918] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:38:55.032101 3599 log.cc:1196] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/c9413670e75d4249bfe5ac6cdb384b30
I20260807 08:38:55.032500 3599 ts_tablet_manager.cc:1939] T c9413670e75d4249bfe5ac6cdb384b30 P 044c79054e9c4acc9f6b8e38d7871f55: Deleting consensus metadata
I20260807 08:38:55.034274 2811 catalog_manager.cc:4841] TS 044c79054e9c4acc9f6b8e38d7871f55 (127.9.174.3:42437): tablet c9413670e75d4249bfe5ac6cdb384b30 (table test-table [id=f473ab98be6f4207bb88be3935f455f3]) successfully deleted
W20260807 08:38:55.037983 2811 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:45383 (error 108)
I20260807 08:38:55.039230 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 3122
I20260807 08:38:55.039510 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:55.039050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:45383 (local address 127.0.0.1:59982)
0807 08:38:55.039232 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:55.039338 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:45383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 08:38:55.047957 2811 catalog_manager.cc:4584] TS 12a78dc0a883421985d3ab03c74e4096 (127.9.174.1:45383): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c9413670e75d4249bfe5ac6cdb384b30: Network error: Client connection negotiation failed: client connection to 127.9.174.1:45383: connect: Connection refused (error 111)
I20260807 08:38:55.072016 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 3335
I20260807 08:38:55.098961 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 2779
2026-08-07T08:38:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2917 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T08:38:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:38:55Z Disabled control of system clock
I20260807 08:38:55.148059 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:43025
--webserver_interface=127.9.174.62
--webserver_port=0
--builtin_ntp_servers=127.9.174.20:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:43025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:38:55.288277 3605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:55.288638 3605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:55.288678 3605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:55.292928 3605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:38:55.293009 3605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:55.293025 3605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:55.293043 3605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:38:55.293059 3605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:38:55.298106 3605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:43025
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:43025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:55.299594 3605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:55.301349 3605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:55.312134 3613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:55.312146 3620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:55.312152 3615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:55.312383 3605 server_base.cc:1034] running on GCE node
I20260807 08:38:55.313138 3605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:55.314517 3605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:55.315716 3605 hybrid_clock.cc:648] HybridClock initialized: now 1786091935315701 us; error 40 us; skew 500 ppm
I20260807 08:38:55.318089 3605 webserver.cc:460] Webserver started at http://127.9.174.62:46573/ using document root <none> and password file <none>
I20260807 08:38:55.318835 3605 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:55.318943 3605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:55.319273 3605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:55.321839 3605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "2a4b7cb5d90f4b0d8db223b38e558f15"
format_stamp: "Formatted at 2026-08-07 08:38:55 on dist-test-slave-h3ft"
server_key: "3605a29a72f9ba4591ad09bfa501c64d"
server_key_iv: "a95e83973165f29c1fed3a27e1b9f8b5"
server_key_version: "encryptionkey@0"
I20260807 08:38:55.322516 3605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "2a4b7cb5d90f4b0d8db223b38e558f15"
format_stamp: "Formatted at 2026-08-07 08:38:55 on dist-test-slave-h3ft"
server_key: "3605a29a72f9ba4591ad09bfa501c64d"
server_key_iv: "a95e83973165f29c1fed3a27e1b9f8b5"
server_key_version: "encryptionkey@0"
I20260807 08:38:55.327241 3605 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 08:38:55.330358 3627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:55.332064 3605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:38:55.332232 3605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "2a4b7cb5d90f4b0d8db223b38e558f15"
format_stamp: "Formatted at 2026-08-07 08:38:55 on dist-test-slave-h3ft"
server_key: "3605a29a72f9ba4591ad09bfa501c64d"
server_key_iv: "a95e83973165f29c1fed3a27e1b9f8b5"
server_key_version: "encryptionkey@0"
I20260807 08:38:55.332381 3605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:55.367089 3605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:55.368202 3605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:55.368490 3605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:55.377375 3605 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:43025
I20260807 08:38:55.377414 3706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:43025 every 8 connection(s)
I20260807 08:38:55.378548 3605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:38:55.382452 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:55.387427 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 3605
I20260807 08:38:55.387604 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:38:55.387997 9912 external_mini_cluster.cc:1428] Setting key 1c2f88b058d3906fbb8723958f2bec67
I20260807 08:38:55.390162 3707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15: Bootstrap starting.
I20260807 08:38:55.393074 3707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:55.393996 3707 log.cc:824] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:55.396322 3707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15: No bootstrap required, opened a new log
I20260807 08:38:55.399399 3710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:55.389921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35822 (local address 127.9.174.62:43025)
0807 08:38:55.390782 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:55.390803 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:55.390846 (+ 43us) server_negotiation.cc:407] Connection header received
0807 08:38:55.391698 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:55.391720 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:55.392228 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:55.392642 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:55.393205 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:55.394339 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:55.395342 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:55.395857 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:55.396064 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:55.397132 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:55.397138 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:55.397152 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:38:55.397609 (+ 457us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:55.397673 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:38:55.397695 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:55.397811 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:55.397981 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 08:38:55.398276 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":163,"threads_started":1}
I20260807 08:38:55.400179 3707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4b7cb5d90f4b0d8db223b38e558f15" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43025 } }
I20260807 08:38:55.400489 3707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:55.400557 3707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4b7cb5d90f4b0d8db223b38e558f15, State: Initialized, Role: FOLLOWER
I20260807 08:38:55.401077 3707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4b7cb5d90f4b0d8db223b38e558f15" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43025 } }
I20260807 08:38:55.401217 3707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:38:55.401307 3707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:38:55.401435 3707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:55.402678 3707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4b7cb5d90f4b0d8db223b38e558f15" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43025 } }
I20260807 08:38:55.403115 3707 leader_election.cc:302] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a4b7cb5d90f4b0d8db223b38e558f15; no voters:
I20260807 08:38:55.403491 3707 leader_election.cc:288] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:38:55.403650 3712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:38:55.403921 3712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [term 1 LEADER]: Becoming Leader. State: Replica: 2a4b7cb5d90f4b0d8db223b38e558f15, State: Running, Role: LEADER
I20260807 08:38:55.404273 3712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4b7cb5d90f4b0d8db223b38e558f15" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43025 } }
I20260807 08:38:55.404893 3707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:38:55.406200 3716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a4b7cb5d90f4b0d8db223b38e558f15. Latest consensus state: current_term: 1 leader_uuid: "2a4b7cb5d90f4b0d8db223b38e558f15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4b7cb5d90f4b0d8db223b38e558f15" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43025 } } }
I20260807 08:38:55.406359 3716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:55.406682 3714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a4b7cb5d90f4b0d8db223b38e558f15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4b7cb5d90f4b0d8db223b38e558f15" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43025 } } }
I20260807 08:38:55.406800 3714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15 [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:55.406919 3725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:38:55.410576 3725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:38:55.416815 3725 catalog_manager.cc:1347] Generated new cluster ID: 4e33f4c14f5f401ca2f7dfaf35638610
I20260807 08:38:55.416896 3725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:38:55.425246 3725 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:38:55.426384 3725 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:38:55.436580 3725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15: Generated new TSK 0
I20260807 08:38:55.437578 3725 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:38:55.446398 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43025
--builtin_ntp_servers=127.9.174.20:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:38:55.585093 3744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:55.585490 3744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:55.585536 3744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:55.590010 3744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:55.590114 3744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:55.590178 3744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:38:55.595667 3744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:55.597249 3744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:55.599120 3744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:55.611374 3756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:55.611374 3752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:55.611379 3753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:55.611982 3744 server_base.cc:1034] running on GCE node
I20260807 08:38:55.612483 3744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:55.613420 3744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:55.614604 3744 hybrid_clock.cc:648] HybridClock initialized: now 1786091935614581 us; error 35 us; skew 500 ppm
I20260807 08:38:55.617132 3744 webserver.cc:460] Webserver started at http://127.9.174.1:37209/ using document root <none> and password file <none>
I20260807 08:38:55.617879 3744 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:55.617950 3744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:55.618266 3744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:55.621112 3744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "4054a72ebaad4c0eb08dc61531bf8d3e"
format_stamp: "Formatted at 2026-08-07 08:38:55 on dist-test-slave-h3ft"
server_key: "e091f0d7037843538c1c5dd2d36e3a06"
server_key_iv: "583ffe299fddcf83063706ebc9553be4"
server_key_version: "encryptionkey@0"
I20260807 08:38:55.621865 3744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "4054a72ebaad4c0eb08dc61531bf8d3e"
format_stamp: "Formatted at 2026-08-07 08:38:55 on dist-test-slave-h3ft"
server_key: "e091f0d7037843538c1c5dd2d36e3a06"
server_key_iv: "583ffe299fddcf83063706ebc9553be4"
server_key_version: "encryptionkey@0"
I20260807 08:38:55.627274 3744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260807 08:38:55.630270 3766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:55.631832 3744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:38:55.631970 3744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "4054a72ebaad4c0eb08dc61531bf8d3e"
format_stamp: "Formatted at 2026-08-07 08:38:55 on dist-test-slave-h3ft"
server_key: "e091f0d7037843538c1c5dd2d36e3a06"
server_key_iv: "583ffe299fddcf83063706ebc9553be4"
server_key_version: "encryptionkey@0"
I20260807 08:38:55.632174 3744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:55.643404 3744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:55.644379 3744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:55.644637 3744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:55.645421 3744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:55.646725 3744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:55.646780 3744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:55.646854 3744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:55.646888 3744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:55.658674 3744 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:35439
I20260807 08:38:55.658763 3937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:35439 every 8 connection(s)
I20260807 08:38:55.659983 3744 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:38:55.667210 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 3744
I20260807 08:38:55.667382 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:38:55.667829 9912 external_mini_cluster.cc:1428] Setting key cabbdafd29526979a63677f8f944102c
I20260807 08:38:55.670251 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43025
--builtin_ntp_servers=127.9.174.20:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:38:55.673026 3710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:55.662512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:60313 (local address 127.9.174.62:43025)
0807 08:38:55.662727 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:55.662732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:55.664296 (+ 1564us) server_negotiation.cc:407] Connection header received
0807 08:38:55.666755 (+ 2459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:55.666759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:55.666844 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:55.666961 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:55.668291 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:55.669034 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:55.670341 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:55.670664 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:55.670785 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:55.672290 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:55.672294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:55.672297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:38:55.672315 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:55.672343 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:38:55.672352 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:55.672445 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:55.672757 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 08:38:55.672857 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 08:38:55.674501 3940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:55.663122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43025 (local address 127.9.174.1:60313)
0807 08:38:55.664070 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:55.664143 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:55.666411 (+ 2268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:55.667217 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:55.667228 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:55.667360 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:55.668116 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:55.668132 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:55.669175 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:55.669178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:55.670192 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:55.670202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:55.670846 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:55.670850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:55.671134 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:55.671954 (+ 820us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:38:55.671956 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:55.672009 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:38:55.672093 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:55.672462 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:55.672489 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:55.672496 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 08:38:55.672689 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 08:38:55.673101 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":146,"threads_started":1}
I20260807 08:38:55.676098 3938 heartbeater.cc:347] Connected to a master server at 127.9.174.62:43025
I20260807 08:38:55.676432 3938 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:55.677299 3938 heartbeater.cc:511] Master 127.9.174.62:43025 requested a full tablet report, sending...
I20260807 08:38:55.679888 3649 ts_manager.cc:194] Registered new tserver with Master: 4054a72ebaad4c0eb08dc61531bf8d3e (127.9.174.1:35439)
I20260807 08:38:55.682070 3649 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.174.1:60313
W20260807 08:38:55.812211 3942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:55.812547 3942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:55.812881 3942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:55.817945 3942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:55.818027 3942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:55.818094 3942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:38:55.823560 3942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:55.824973 3942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:55.826680 3942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:55.838411 3961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:55.838418 3959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:55.838445 3954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:55.838872 3942 server_base.cc:1034] running on GCE node
I20260807 08:38:55.839480 3942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:55.840222 3942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:55.841435 3942 hybrid_clock.cc:648] HybridClock initialized: now 1786091935841401 us; error 68 us; skew 500 ppm
I20260807 08:38:55.843876 3942 webserver.cc:460] Webserver started at http://127.9.174.2:37445/ using document root <none> and password file <none>
I20260807 08:38:55.844586 3942 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:55.844655 3942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:55.844861 3942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:55.847311 3942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "da974d2039b146e2b184a9458686d496"
format_stamp: "Formatted at 2026-08-07 08:38:55 on dist-test-slave-h3ft"
server_key: "65b6c673d2c8a04cb603b9c69932c1d8"
server_key_iv: "abc4e01adc80a9860d1bd33dba11380a"
server_key_version: "encryptionkey@0"
I20260807 08:38:55.847944 3942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "da974d2039b146e2b184a9458686d496"
format_stamp: "Formatted at 2026-08-07 08:38:55 on dist-test-slave-h3ft"
server_key: "65b6c673d2c8a04cb603b9c69932c1d8"
server_key_iv: "abc4e01adc80a9860d1bd33dba11380a"
server_key_version: "encryptionkey@0"
I20260807 08:38:55.852113 3942 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:38:55.854748 3971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:55.856267 3942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:38:55.856422 3942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "da974d2039b146e2b184a9458686d496"
format_stamp: "Formatted at 2026-08-07 08:38:55 on dist-test-slave-h3ft"
server_key: "65b6c673d2c8a04cb603b9c69932c1d8"
server_key_iv: "abc4e01adc80a9860d1bd33dba11380a"
server_key_version: "encryptionkey@0"
I20260807 08:38:55.856542 3942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:55.889884 3942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:55.890858 3942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:55.891139 3942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:55.891986 3942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:55.893191 3942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:55.893244 3942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:55.893324 3942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:55.893368 3942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:55.904825 3942 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:42321
I20260807 08:38:55.904907 4143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:42321 every 8 connection(s)
I20260807 08:38:55.906459 3942 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:38:55.912055 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 3942
I20260807 08:38:55.912202 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:38:55.912645 9912 external_mini_cluster.cc:1428] Setting key 4f9cec59f8e28a669c2993ecb318ebf2
I20260807 08:38:55.915489 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43025
--builtin_ntp_servers=127.9.174.20:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:38:55.919489 3710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:55.908928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:39015 (local address 127.9.174.62:43025)
0807 08:38:55.909154 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:55.909158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:55.910541 (+ 1383us) server_negotiation.cc:407] Connection header received
0807 08:38:55.912509 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:55.912513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:55.912596 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:55.912711 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:55.914214 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:55.914953 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:55.916846 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:55.917076 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:55.917165 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:55.918706 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:55.918713 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:55.918718 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:38:55.918736 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:55.918767 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:38:55.918781 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:55.918856 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:55.919162 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 08:38:55.919249 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260807 08:38:55.920516 4148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:55.909434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43025 (local address 127.9.174.2:39015)
0807 08:38:55.910359 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:55.910410 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:55.912190 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:55.912947 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:55.912957 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:55.913186 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:55.914032 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:55.914050 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:55.915148 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:55.915154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:55.916711 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:55.916727 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:55.917252 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:55.917258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:55.917517 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:55.918398 (+ 881us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:38:55.918399 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:55.918454 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:38:55.918533 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:55.918899 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:55.918913 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:55.918923 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 08:38:55.919138 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 08:38:55.919425 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":160,"threads_started":1}
I20260807 08:38:55.921978 4144 heartbeater.cc:347] Connected to a master server at 127.9.174.62:43025
I20260807 08:38:55.922271 4144 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:55.922912 4144 heartbeater.cc:511] Master 127.9.174.62:43025 requested a full tablet report, sending...
I20260807 08:38:55.924237 3649 ts_manager.cc:194] Registered new tserver with Master: da974d2039b146e2b184a9458686d496 (127.9.174.2:42321)
I20260807 08:38:55.925223 3649 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.174.2:39015
W20260807 08:38:56.050637 4149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:56.050992 4149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:56.051101 4149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:56.055375 4149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:56.055456 4149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:56.055521 4149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:38:56.060827 4149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:56.062453 4149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:56.064419 4149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:56.076509 4154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:56.076654 4149 server_base.cc:1034] running on GCE node
W20260807 08:38:56.076515 4157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:56.076655 4155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:56.077397 4149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:56.078255 4149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:56.079433 4149 hybrid_clock.cc:648] HybridClock initialized: now 1786091936079417 us; error 31 us; skew 500 ppm
I20260807 08:38:56.081951 4149 webserver.cc:460] Webserver started at http://127.9.174.3:44853/ using document root <none> and password file <none>
I20260807 08:38:56.082731 4149 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:56.082808 4149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:56.083150 4149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:56.085825 4149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "032c3826cc0a4649a38c98ddd1cd9e05"
format_stamp: "Formatted at 2026-08-07 08:38:56 on dist-test-slave-h3ft"
server_key: "ba9c3f016fe84ca45dee3c8f5e8f0e44"
server_key_iv: "887ac495452f40303b8c0e3fb90a8b88"
server_key_version: "encryptionkey@0"
I20260807 08:38:56.086548 4149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "032c3826cc0a4649a38c98ddd1cd9e05"
format_stamp: "Formatted at 2026-08-07 08:38:56 on dist-test-slave-h3ft"
server_key: "ba9c3f016fe84ca45dee3c8f5e8f0e44"
server_key_iv: "887ac495452f40303b8c0e3fb90a8b88"
server_key_version: "encryptionkey@0"
I20260807 08:38:56.091313 4149 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 08:38:56.094499 4165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:56.096300 4149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:38:56.096488 4149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "032c3826cc0a4649a38c98ddd1cd9e05"
format_stamp: "Formatted at 2026-08-07 08:38:56 on dist-test-slave-h3ft"
server_key: "ba9c3f016fe84ca45dee3c8f5e8f0e44"
server_key_iv: "887ac495452f40303b8c0e3fb90a8b88"
server_key_version: "encryptionkey@0"
I20260807 08:38:56.096647 4149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:56.114020 4149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:56.115088 4149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:56.115340 4149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:56.116132 4149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:56.117390 4149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:56.117470 4149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:56.117552 4149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:56.117601 4149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:56.129532 4149 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:35095
I20260807 08:38:56.129566 4333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:35095 every 8 connection(s)
I20260807 08:38:56.130736 4149 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:38:56.136420 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 4149
I20260807 08:38:56.136628 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:38:56.137038 9912 external_mini_cluster.cc:1428] Setting key 90b6152b45c2668e77c416a574a5246e
I20260807 08:38:56.143085 3710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:56.133336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:36023 (local address 127.9.174.62:43025)
0807 08:38:56.133563 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:56.133568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:56.134988 (+ 1420us) server_negotiation.cc:407] Connection header received
0807 08:38:56.137262 (+ 2274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:56.137269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:56.137363 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:56.137501 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:56.138795 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.139448 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.140837 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.141081 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.141198 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:56.142475 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:56.142479 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:56.142482 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:38:56.142501 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:56.142525 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:38:56.142534 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:56.142617 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:56.142791 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 08:38:56.142864 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 08:38:56.144168 4336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:56.133822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43025 (local address 127.9.174.3:36023)
0807 08:38:56.134775 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:56.134828 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:56.136891 (+ 2063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:56.137781 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:56.137789 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:56.137894 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:56.138634 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:56.138653 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.139586 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.139589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:56.140689 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:56.140699 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.141249 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.141253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:56.141447 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:56.142179 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:38:56.142180 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:56.142229 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:38:56.142307 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:56.142620 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:56.142633 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:56.142640 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 08:38:56.142791 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 08:38:56.143162 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":157,"threads_started":1}
I20260807 08:38:56.145443 4334 heartbeater.cc:347] Connected to a master server at 127.9.174.62:43025
I20260807 08:38:56.145749 4334 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:56.146404 4334 heartbeater.cc:511] Master 127.9.174.62:43025 requested a full tablet report, sending...
I20260807 08:38:56.147730 3649 ts_manager.cc:194] Registered new tserver with Master: 032c3826cc0a4649a38c98ddd1cd9e05 (127.9.174.3:35095)
I20260807 08:38:56.148375 3649 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.174.3:36023
I20260807 08:38:56.151823 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:38:56.158656 3710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:56.154448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35830 (local address 127.9.174.62:43025)
0807 08:38:56.154600 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:56.154604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:56.154774 (+ 170us) server_negotiation.cc:407] Connection header received
0807 08:38:56.154908 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:56.154910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:56.154959 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:56.155130 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:56.155422 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.155986 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.156877 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.157132 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.157216 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:56.158220 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:56.158223 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:56.158226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:38:56.158239 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:56.158262 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:38:56.158270 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:56.158331 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:56.158436 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 08:38:56.158501 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 08:38:56.162410 3649 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:38:56.166373 3649 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:38:56.183741 4354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:56.177206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:35095 (local address 127.0.0.1:60208)
0807 08:38:56.178101 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:56.178166 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:56.178322 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:56.179227 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:56.179233 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:56.179258 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:56.179448 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:56.179457 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.181059 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.181063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:56.181997 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:56.182006 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.183123 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.183127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:56.183241 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:56.183283 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 08:38:56.183393 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 08:38:56.183544 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"mutex_wait_us":22,"thread_start_us":108,"threads_started":1}
I20260807 08:38:56.183775 4355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:56.177567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60208 (local address 127.9.174.3:35095)
0807 08:38:56.178626 (+ 1059us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:56.178638 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:56.178668 (+ 30us) server_negotiation.cc:407] Connection header received
0807 08:38:56.178761 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:56.178766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:56.179045 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:56.179220 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:56.179567 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.180899 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.182123 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.182985 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.183126 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:56.183370 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:56.183434 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 08:38:56.183549 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":936,"thread_start_us":215,"threads_started":1}
I20260807 08:38:56.184266 4356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:56.177591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:35439 (local address 127.0.0.1:36638)
0807 08:38:56.178709 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:56.178723 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:56.178856 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:56.179442 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:56.179444 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:56.179461 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:56.179590 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:56.179597 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.180753 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.180760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:56.182101 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:56.182112 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.183443 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.183446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:56.183551 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:56.183575 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 08:38:56.183649 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 08:38:56.183722 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1044,"mutex_wait_us":93,"spinlock_wait_cycles":4736,"thread_start_us":96,"threads_started":1}
I20260807 08:38:56.184561 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:56.177206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:42321 (local address 127.0.0.1:42960)
0807 08:38:56.178724 (+ 1518us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:56.178738 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:56.178856 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:56.179600 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:56.179603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:56.179619 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:56.179712 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:56.179717 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.181163 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.181166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:56.182725 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:56.182734 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.184134 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.184139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:56.184239 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:56.184262 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:38:56.184338 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 08:38:56.184415 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1469,"mutex_wait_us":771,"spinlock_wait_cycles":5760,"thread_start_us":112,"threads_started":1}
I20260807 08:38:56.184783 4357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:56.178227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36638 (local address 127.9.174.1:35439)
0807 08:38:56.178626 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:56.178635 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:56.178910 (+ 275us) server_negotiation.cc:407] Connection header received
0807 08:38:56.179033 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:56.179038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:56.179230 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:56.179362 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:56.179715 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.180564 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.182216 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.182993 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.183125 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:56.183370 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:56.183824 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 08:38:56.184612 (+ 788us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":155,"threads_started":1}
I20260807 08:38:56.185132 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:56.177418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42960 (local address 127.9.174.2:42321)
0807 08:38:56.178366 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:56.178382 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:56.178853 (+ 471us) server_negotiation.cc:407] Connection header received
0807 08:38:56.179095 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:56.179100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:56.179312 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:56.179473 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:56.179851 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.181042 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.182859 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.183721 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.183834 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:56.184063 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:56.184828 (+ 765us) server_negotiation.cc:299] Negotiation successful
0807 08:38:56.184958 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":816,"thread_start_us":135,"threads_started":1}
I20260807 08:38:56.187760 3835 tablet_service.cc:1467] Processing CreateTablet for tablet 8a886500972243aeb44330682ce701b1 (DEFAULT_TABLE table=test-table [id=7af1d7cd86794ade8df3742141704785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:38:56.187764 4039 tablet_service.cc:1467] Processing CreateTablet for tablet 8a886500972243aeb44330682ce701b1 (DEFAULT_TABLE table=test-table [id=7af1d7cd86794ade8df3742141704785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:38:56.187767 4236 tablet_service.cc:1467] Processing CreateTablet for tablet 8a886500972243aeb44330682ce701b1 (DEFAULT_TABLE table=test-table [id=7af1d7cd86794ade8df3742141704785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:38:56.188985 3835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a886500972243aeb44330682ce701b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:56.188993 4039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a886500972243aeb44330682ce701b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:56.188988 4236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a886500972243aeb44330682ce701b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:56.197157 4358 tablet_bootstrap.cc:492] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e: Bootstrap starting.
I20260807 08:38:56.198439 4361 tablet_bootstrap.cc:492] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496: Bootstrap starting.
I20260807 08:38:56.199386 4362 tablet_bootstrap.cc:492] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05: Bootstrap starting.
I20260807 08:38:56.200731 4358 tablet_bootstrap.cc:654] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:56.200776 4361 tablet_bootstrap.cc:654] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:56.201777 4362 tablet_bootstrap.cc:654] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:56.201768 4358 log.cc:824] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:56.201781 4361 log.cc:824] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:56.202684 4362 log.cc:824] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:56.204383 4361 tablet_bootstrap.cc:492] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496: No bootstrap required, opened a new log
I20260807 08:38:56.204420 4358 tablet_bootstrap.cc:492] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e: No bootstrap required, opened a new log
I20260807 08:38:56.204643 4361 ts_tablet_manager.cc:1397] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 08:38:56.204646 4358 ts_tablet_manager.cc:1397] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 08:38:56.204742 4362 tablet_bootstrap.cc:492] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05: No bootstrap required, opened a new log
I20260807 08:38:56.204921 4362 ts_tablet_manager.cc:1397] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260807 08:38:56.209152 4361 raft_consensus.cc:348] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da974d2039b146e2b184a9458686d496" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 42321 } } peers { permanent_uuid: "032c3826cc0a4649a38c98ddd1cd9e05" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35095 } } peers { permanent_uuid: "4054a72ebaad4c0eb08dc61531bf8d3e" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 35439 } }
I20260807 08:38:56.209460 4361 raft_consensus.cc:374] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:56.209528 4361 raft_consensus.cc:729] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da974d2039b146e2b184a9458686d496, State: Initialized, Role: FOLLOWER
I20260807 08:38:56.210048 4361 consensus_queue.cc:260] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da974d2039b146e2b184a9458686d496" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 42321 } } peers { permanent_uuid: "032c3826cc0a4649a38c98ddd1cd9e05" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35095 } } peers { permanent_uuid: "4054a72ebaad4c0eb08dc61531bf8d3e" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 35439 } }
I20260807 08:38:56.210047 4362 raft_consensus.cc:348] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da974d2039b146e2b184a9458686d496" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 42321 } } peers { permanent_uuid: "032c3826cc0a4649a38c98ddd1cd9e05" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35095 } } peers { permanent_uuid: "4054a72ebaad4c0eb08dc61531bf8d3e" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 35439 } }
I20260807 08:38:56.210004 4358 raft_consensus.cc:348] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da974d2039b146e2b184a9458686d496" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 42321 } } peers { permanent_uuid: "032c3826cc0a4649a38c98ddd1cd9e05" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35095 } } peers { permanent_uuid: "4054a72ebaad4c0eb08dc61531bf8d3e" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 35439 } }
I20260807 08:38:56.210276 4358 raft_consensus.cc:374] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:56.210276 4362 raft_consensus.cc:374] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:56.210328 4362 raft_consensus.cc:729] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 032c3826cc0a4649a38c98ddd1cd9e05, State: Initialized, Role: FOLLOWER
I20260807 08:38:56.210337 4358 raft_consensus.cc:729] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4054a72ebaad4c0eb08dc61531bf8d3e, State: Initialized, Role: FOLLOWER
I20260807 08:38:56.210777 4358 consensus_queue.cc:260] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da974d2039b146e2b184a9458686d496" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 42321 } } peers { permanent_uuid: "032c3826cc0a4649a38c98ddd1cd9e05" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35095 } } peers { permanent_uuid: "4054a72ebaad4c0eb08dc61531bf8d3e" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 35439 } }
I20260807 08:38:56.210824 4362 consensus_queue.cc:260] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da974d2039b146e2b184a9458686d496" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 42321 } } peers { permanent_uuid: "032c3826cc0a4649a38c98ddd1cd9e05" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35095 } } peers { permanent_uuid: "4054a72ebaad4c0eb08dc61531bf8d3e" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 35439 } }
I20260807 08:38:56.211485 4361 ts_tablet_manager.cc:1428] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 08:38:56.211664 4334 heartbeater.cc:503] Master 127.9.174.62:43025 was elected leader, sending a full tablet report...
I20260807 08:38:56.211815 4362 ts_tablet_manager.cc:1428] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20260807 08:38:56.211814 4358 ts_tablet_manager.cc:1428] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20260807 08:38:56.211944 4144 heartbeater.cc:503] Master 127.9.174.62:43025 was elected leader, sending a full tablet report...
I20260807 08:38:56.212170 3938 heartbeater.cc:503] Master 127.9.174.62:43025 was elected leader, sending a full tablet report...
I20260807 08:38:56.325790 4368 raft_consensus.cc:482] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:38:56.326170 4368 raft_consensus.cc:504] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da974d2039b146e2b184a9458686d496" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 42321 } } peers { permanent_uuid: "032c3826cc0a4649a38c98ddd1cd9e05" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35095 } } peers { permanent_uuid: "4054a72ebaad4c0eb08dc61531bf8d3e" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 35439 } }
I20260807 08:38:56.327831 4368 leader_election.cc:288] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 032c3826cc0a4649a38c98ddd1cd9e05 (127.9.174.3:35095), 4054a72ebaad4c0eb08dc61531bf8d3e (127.9.174.1:35439)
I20260807 08:38:56.332322 4148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:56.328146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:35095 (local address 127.9.174.2:57341)
0807 08:38:56.328302 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:56.328323 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:56.328454 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:56.328902 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:56.328905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:56.328924 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:56.329072 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:56.329078 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.329886 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.329889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:56.330768 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:56.330778 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.331810 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.331813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:56.331922 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:56.331950 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 08:38:56.332029 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 08:38:56.332118 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":3}
I20260807 08:38:56.332361 4355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:56.328422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:57341 (local address 127.9.174.3:35095)
0807 08:38:56.328650 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:56.328654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:56.328668 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:38:56.328734 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:56.328736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:56.328784 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:56.328881 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:56.329203 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.329762 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.330935 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.331636 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.331750 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:56.331829 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:56.332099 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 08:38:56.332199 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260807 08:38:56.332518 4357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:56.328389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:41811 (local address 127.9.174.1:35439)
0807 08:38:56.328584 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:56.328590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:56.328832 (+ 242us) server_negotiation.cc:407] Connection header received
0807 08:38:56.329013 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:56.329016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:56.329072 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:56.329182 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:56.329499 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.330031 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.331192 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.331836 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.331950 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:56.332020 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:56.332274 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 08:38:56.332381 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 08:38:56.332885 4371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:56.328188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:35439 (local address 127.9.174.2:41811)
0807 08:38:56.328713 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:56.328724 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:56.328866 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:56.329189 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:56.329191 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:56.329208 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:56.329370 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:56.329376 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.330155 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.330160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:56.331058 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:56.331070 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.331990 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.331994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:56.332123 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:56.332147 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 08:38:56.332227 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 08:38:56.332294 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":19,"thread_start_us":178,"threads_started":1}
I20260807 08:38:56.333386 4265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a886500972243aeb44330682ce701b1" candidate_uuid: "da974d2039b146e2b184a9458686d496" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "032c3826cc0a4649a38c98ddd1cd9e05" is_pre_election: true
I20260807 08:38:56.333599 3867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a886500972243aeb44330682ce701b1" candidate_uuid: "da974d2039b146e2b184a9458686d496" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4054a72ebaad4c0eb08dc61531bf8d3e" is_pre_election: true
I20260807 08:38:56.333848 4265 raft_consensus.cc:2454] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da974d2039b146e2b184a9458686d496 in term 0.
I20260807 08:38:56.333881 3867 raft_consensus.cc:2454] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da974d2039b146e2b184a9458686d496 in term 0.
I20260807 08:38:56.334482 3977 leader_election.cc:302] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4054a72ebaad4c0eb08dc61531bf8d3e, da974d2039b146e2b184a9458686d496; no voters:
I20260807 08:38:56.334796 4368 raft_consensus.cc:2781] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:38:56.334894 4368 raft_consensus.cc:482] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:38:56.334939 4368 raft_consensus.cc:3037] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:56.336338 4368 raft_consensus.cc:504] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da974d2039b146e2b184a9458686d496" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 42321 } } peers { permanent_uuid: "032c3826cc0a4649a38c98ddd1cd9e05" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35095 } } peers { permanent_uuid: "4054a72ebaad4c0eb08dc61531bf8d3e" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 35439 } }
I20260807 08:38:56.336858 4368 leader_election.cc:288] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [CANDIDATE]: Term 1 election: Requested vote from peers 032c3826cc0a4649a38c98ddd1cd9e05 (127.9.174.3:35095), 4054a72ebaad4c0eb08dc61531bf8d3e (127.9.174.1:35439)
I20260807 08:38:56.337311 4265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a886500972243aeb44330682ce701b1" candidate_uuid: "da974d2039b146e2b184a9458686d496" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "032c3826cc0a4649a38c98ddd1cd9e05"
I20260807 08:38:56.337308 3867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a886500972243aeb44330682ce701b1" candidate_uuid: "da974d2039b146e2b184a9458686d496" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4054a72ebaad4c0eb08dc61531bf8d3e"
I20260807 08:38:56.337502 3867 raft_consensus.cc:3037] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:56.337517 4265 raft_consensus.cc:3037] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:56.338860 4265 raft_consensus.cc:2454] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da974d2039b146e2b184a9458686d496 in term 1.
I20260807 08:38:56.338912 3867 raft_consensus.cc:2454] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da974d2039b146e2b184a9458686d496 in term 1.
I20260807 08:38:56.339268 3977 leader_election.cc:302] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4054a72ebaad4c0eb08dc61531bf8d3e, da974d2039b146e2b184a9458686d496; no voters:
I20260807 08:38:56.339537 4368 raft_consensus.cc:2781] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:38:56.339805 4368 raft_consensus.cc:686] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [term 1 LEADER]: Becoming Leader. State: Replica: da974d2039b146e2b184a9458686d496, State: Running, Role: LEADER
I20260807 08:38:56.340212 4368 consensus_queue.cc:237] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da974d2039b146e2b184a9458686d496" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 42321 } } peers { permanent_uuid: "032c3826cc0a4649a38c98ddd1cd9e05" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35095 } } peers { permanent_uuid: "4054a72ebaad4c0eb08dc61531bf8d3e" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 35439 } }
I20260807 08:38:56.344169 3646 catalog_manager.cc:5495] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 reported cstate change: term changed from 0 to 1, leader changed from <none> to da974d2039b146e2b184a9458686d496 (127.9.174.2). New cstate: current_term: 1 leader_uuid: "da974d2039b146e2b184a9458686d496" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da974d2039b146e2b184a9458686d496" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "032c3826cc0a4649a38c98ddd1cd9e05" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4054a72ebaad4c0eb08dc61531bf8d3e" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 35439 } health_report { overall_health: UNKNOWN } } }
I20260807 08:38:56.380348 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:56.376351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42962 (local address 127.9.174.2:42321)
0807 08:38:56.376573 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:56.376577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:56.376590 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:38:56.376692 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:56.376695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:56.376767 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:56.376897 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:38:56.377219 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.377784 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.378694 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:56.378928 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:56.379067 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:56.379426 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:38:56.379899 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:38:56.380041 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:56.380085 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 08:38:56.380150 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260807 08:38:56.382061 4335 tablet.cc:2369] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:38:56.388536 3867 raft_consensus.cc:1261] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e [term 1 FOLLOWER]: Refusing update from remote peer da974d2039b146e2b184a9458686d496: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:38:56.388537 4265 raft_consensus.cc:1261] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05 [term 1 FOLLOWER]: Refusing update from remote peer da974d2039b146e2b184a9458686d496: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:38:56.389407 4368 consensus_queue.cc:1035] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [LEADER]: Connected to new peer: Peer: permanent_uuid: "032c3826cc0a4649a38c98ddd1cd9e05" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 35095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:38:56.389674 4374 consensus_queue.cc:1035] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4054a72ebaad4c0eb08dc61531bf8d3e" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 35439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:38:56.410243 3645 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:38:56.410465 3645 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:38:56.413808 3645 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a4b7cb5d90f4b0d8db223b38e558f15: Sending DeleteTablet for 3 replicas of tablet 8a886500972243aeb44330682ce701b1
I20260807 08:38:56.414958 4039 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a886500972243aeb44330682ce701b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:38:56 UTC) from {username='slave'} at 127.0.0.1:42960
I20260807 08:38:56.415117 4236 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a886500972243aeb44330682ce701b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:38:56 UTC) from {username='slave'} at 127.0.0.1:60208
I20260807 08:38:56.415234 3835 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a886500972243aeb44330682ce701b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:38:56 UTC) from {username='slave'} at 127.0.0.1:36638
I20260807 08:38:56.415771 4398 tablet_replica.cc:330] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05: stopping tablet replica
I20260807 08:38:56.416000 4399 tablet_replica.cc:330] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e: stopping tablet replica
I20260807 08:38:56.416206 4398 raft_consensus.cc:2229] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:38:56.416249 4397 tablet_replica.cc:330] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496: stopping tablet replica
I20260807 08:38:56.416495 4399 raft_consensus.cc:2229] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:38:56.416519 4397 raft_consensus.cc:2229] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:38:56.416642 4398 raft_consensus.cc:2258] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:38:56.416862 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 3744
I20260807 08:38:56.416955 4397 raft_consensus.cc:2258] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:38:56.416996 4399 raft_consensus.cc:2258] T 8a886500972243aeb44330682ce701b1 P 4054a72ebaad4c0eb08dc61531bf8d3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:38:56.422585 4397 ts_tablet_manager.cc:1905] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:38:56.426226 4397 ts_tablet_manager.cc:1918] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:38:56.426506 4397 log.cc:1196] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/8a886500972243aeb44330682ce701b1
I20260807 08:38:56.426981 4397 ts_tablet_manager.cc:1939] T 8a886500972243aeb44330682ce701b1 P da974d2039b146e2b184a9458686d496: Deleting consensus metadata
I20260807 08:38:56.428478 4398 ts_tablet_manager.cc:1905] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:38:56.428640 3629 catalog_manager.cc:4841] TS da974d2039b146e2b184a9458686d496 (127.9.174.2:42321): tablet 8a886500972243aeb44330682ce701b1 (table test-table [id=7af1d7cd86794ade8df3742141704785]) successfully deleted
W20260807 08:38:56.431906 3631 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:35439 (error 108)
I20260807 08:38:56.432389 4398 ts_tablet_manager.cc:1918] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:38:56.432523 4398 log.cc:1196] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/8a886500972243aeb44330682ce701b1
I20260807 08:38:56.432937 4398 ts_tablet_manager.cc:1939] T 8a886500972243aeb44330682ce701b1 P 032c3826cc0a4649a38c98ddd1cd9e05: Deleting consensus metadata
I20260807 08:38:56.433487 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:56.433127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:35439 (local address 127.0.0.1:36648)
0807 08:38:56.433293 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:56.433377 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:35439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 08:38:56.433594 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 3942
W20260807 08:38:56.433796 3631 catalog_manager.cc:4584] TS 4054a72ebaad4c0eb08dc61531bf8d3e (127.9.174.1:35439): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a886500972243aeb44330682ce701b1: Network error: Client connection negotiation failed: client connection to 127.9.174.1:35439: connect: Connection refused (error 111)
I20260807 08:38:56.446344 3630 catalog_manager.cc:4841] TS 032c3826cc0a4649a38c98ddd1cd9e05 (127.9.174.3:35095): tablet 8a886500972243aeb44330682ce701b1 (table test-table [id=7af1d7cd86794ade8df3742141704785]) successfully deleted
I20260807 08:38:56.455430 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 4149
I20260807 08:38:56.471127 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 3605
2026-08-07T08:38:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1371 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:38:56 dist-test-slave-h3ft krb5kdc[4403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:38:56 dist-test-slave-h3ft krb5kdc[4403](info): set up 2 sockets
Aug 07 08:38:56 dist-test-slave-h3ft krb5kdc[4403](info): commencing operation
krb5kdc: starting...
W20260807 08:38:57.752686 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:38:57 dist-test-slave-h3ft krb5kdc[4403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:38:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:38:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:38:57.907392 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:38371
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:46005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:38371
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:38:58.040794 4419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:58.041189 4419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:58.041271 4419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:58.045430 4419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:38:58.045557 4419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:58.045594 4419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:58.045637 4419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:38:58.045677 4419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:38:58.051443 4419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:46005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:38371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:38371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:58.052918 4419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:58.054255 4419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:58.064778 4426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:58.064834 4427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:58.064798 4431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:58.065665 4419 server_base.cc:1034] running on GCE node
I20260807 08:38:58.066313 4419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:58.067658 4419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:58.068852 4419 hybrid_clock.cc:648] HybridClock initialized: now 1786091938068834 us; error 33 us; skew 500 ppm
Aug 07 08:38:58 dist-test-slave-h3ft krb5kdc[4403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091938, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:58.072098 4419 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:38:58.073266 4419 webserver.cc:460] Webserver started at http://127.9.174.62:37957/ using document root <none> and password file <none>
I20260807 08:38:58.073927 4419 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:58.074002 4419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:58.074216 4419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:58.076768 4419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "c0a22be852cb452289a73251be101e6d"
format_stamp: "Formatted at 2026-08-07 08:38:58 on dist-test-slave-h3ft"
server_key: "cee1c58ebf30fa4335dd7a1008c485a2"
server_key_iv: "f6564bfd4ca2fe0d1f80655ac80ce0c5"
server_key_version: "encryptionkey@0"
I20260807 08:38:58.077399 4419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "c0a22be852cb452289a73251be101e6d"
format_stamp: "Formatted at 2026-08-07 08:38:58 on dist-test-slave-h3ft"
server_key: "cee1c58ebf30fa4335dd7a1008c485a2"
server_key_iv: "f6564bfd4ca2fe0d1f80655ac80ce0c5"
server_key_version: "encryptionkey@0"
I20260807 08:38:58.081849 4419 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 08:38:58.084712 4446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:58.086194 4419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 08:38:58.086336 4419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "c0a22be852cb452289a73251be101e6d"
format_stamp: "Formatted at 2026-08-07 08:38:58 on dist-test-slave-h3ft"
server_key: "cee1c58ebf30fa4335dd7a1008c485a2"
server_key_iv: "f6564bfd4ca2fe0d1f80655ac80ce0c5"
server_key_version: "encryptionkey@0"
I20260807 08:38:58.086475 4419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:58.102599 4419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:58.106206 4419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:58.106462 4419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:58.115444 4419 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:38371
I20260807 08:38:58.115463 4515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:38371 every 8 connection(s)
I20260807 08:38:58.116730 4419 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:38:58.117136 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 4419
I20260807 08:38:58.117316 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:38:58.117744 9912 external_mini_cluster.cc:1428] Setting key e4cbefa4951ad0691ff7503a22eeaf88
I20260807 08:38:58.120893 4516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 08:38:58 dist-test-slave-h3ft krb5kdc[4403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:38:58.128077 4516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d: Bootstrap starting.
I20260807 08:38:58.131107 4516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:58.132019 4516 log.cc:824] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:58.133657 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:58.119605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32882 (local address 127.9.174.62:38371)
0807 08:38:58.120149 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:58.120168 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:58.120212 (+ 44us) server_negotiation.cc:407] Connection header received
0807 08:38:58.120969 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:58.120990 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:58.121415 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:58.121812 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:58.122766 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:58.123989 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:58.124933 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:58.125460 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:58.125659 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:58.128609 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:58.128615 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:58.128629 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:58.128661 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:58.130415 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:58.130903 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:58.130909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:58.130914 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:58.131101 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:58.131420 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:58.131423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:58.131425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:58.131895 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:58.132125 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:58.132442 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 08:38:58.132803 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":176,"threads_started":1}
I20260807 08:38:58.134435 4516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d: No bootstrap required, opened a new log
I20260807 08:38:58.138693 4516 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a22be852cb452289a73251be101e6d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38371 } }
I20260807 08:38:58.138970 4516 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:58.139096 4516 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0a22be852cb452289a73251be101e6d, State: Initialized, Role: FOLLOWER
I20260807 08:38:58.139559 4516 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a22be852cb452289a73251be101e6d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38371 } }
I20260807 08:38:58.139722 4516 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:38:58.139822 4516 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:38:58.139946 4516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:58.140954 4516 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a22be852cb452289a73251be101e6d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38371 } }
I20260807 08:38:58.141328 4516 leader_election.cc:302] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0a22be852cb452289a73251be101e6d; no voters:
I20260807 08:38:58.141743 4516 leader_election.cc:288] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:38:58.141873 4527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:38:58.142097 4527 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [term 1 LEADER]: Becoming Leader. State: Replica: c0a22be852cb452289a73251be101e6d, State: Running, Role: LEADER
I20260807 08:38:58.142422 4527 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a22be852cb452289a73251be101e6d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38371 } }
I20260807 08:38:58.143249 4516 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:38:58.144434 4530 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0a22be852cb452289a73251be101e6d. Latest consensus state: current_term: 1 leader_uuid: "c0a22be852cb452289a73251be101e6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a22be852cb452289a73251be101e6d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38371 } } }
I20260807 08:38:58.144578 4530 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:58.144846 4528 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0a22be852cb452289a73251be101e6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a22be852cb452289a73251be101e6d" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38371 } } }
I20260807 08:38:58.144958 4528 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:58.145242 4541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:38:58.148856 4541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:38:58.155059 4541 catalog_manager.cc:1347] Generated new cluster ID: 4d560f238b3948c094908b760914d46a
I20260807 08:38:58.155130 4541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:38:58.177685 4541 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:38:58.179236 4541 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:38:58.190616 4541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d: Generated new TSK 0
I20260807 08:38:58.191697 4541 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:38:58.234623 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:38371
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:46005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:38:58.367166 4556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:58.367533 4556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:58.367623 4556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:58.371826 4556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:58.371943 4556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:58.372059 4556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:38:58.377902 4556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:46005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:38371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:58.379449 4556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:58.380774 4556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:58.392014 4567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:58.392105 4564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:58.392409 4556 server_base.cc:1034] running on GCE node
W20260807 08:38:58.392064 4563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:58.392973 4556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:58.393711 4556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:58.394872 4556 hybrid_clock.cc:648] HybridClock initialized: now 1786091938394848 us; error 38 us; skew 500 ppm
Aug 07 08:38:58 dist-test-slave-h3ft krb5kdc[4403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091938, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:58.398129 4556 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:38:58.399178 4556 webserver.cc:460] Webserver started at http://127.9.174.1:41927/ using document root <none> and password file <none>
I20260807 08:38:58.399842 4556 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:58.399907 4556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:58.400199 4556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:58.402585 4556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "d31d813e1cdc476ca4a40d93e7bd6785"
format_stamp: "Formatted at 2026-08-07 08:38:58 on dist-test-slave-h3ft"
server_key: "85f620002939c4aea644e81892ae00b4"
server_key_iv: "b6616cca364b5ece733cec1568659f20"
server_key_version: "encryptionkey@0"
I20260807 08:38:58.403252 4556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "d31d813e1cdc476ca4a40d93e7bd6785"
format_stamp: "Formatted at 2026-08-07 08:38:58 on dist-test-slave-h3ft"
server_key: "85f620002939c4aea644e81892ae00b4"
server_key_iv: "b6616cca364b5ece733cec1568659f20"
server_key_version: "encryptionkey@0"
I20260807 08:38:58.407557 4556 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:38:58.410362 4576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:58.411796 4556 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:38:58.411924 4556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "d31d813e1cdc476ca4a40d93e7bd6785"
format_stamp: "Formatted at 2026-08-07 08:38:58 on dist-test-slave-h3ft"
server_key: "85f620002939c4aea644e81892ae00b4"
server_key_iv: "b6616cca364b5ece733cec1568659f20"
server_key_version: "encryptionkey@0"
I20260807 08:38:58.412046 4556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:58.424294 4556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:58.427654 4556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:58.427873 4556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:58.428560 4556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:58.429512 4556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:58.429562 4556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:58.429638 4556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:58.429669 4556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:58.440155 4556 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:43949
I20260807 08:38:58.440184 4728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:43949 every 8 connection(s)
I20260807 08:38:58.441172 4556 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:38:58.443461 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 4556
I20260807 08:38:58.443632 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:38:58.444068 9912 external_mini_cluster.cc:1428] Setting key afdc0a2a0313ee848c6ec232b8842a9e
Aug 07 08:38:58 dist-test-slave-h3ft krb5kdc[4403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091938, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
I20260807 08:38:58.458709 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:58.443341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:40153 (local address 127.9.174.62:38371)
0807 08:38:58.443555 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:58.443562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:58.444749 (+ 1187us) server_negotiation.cc:407] Connection header received
0807 08:38:58.446678 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:58.446681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:58.446734 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:58.446806 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:58.448587 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:58.449173 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:58.450925 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:58.451414 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:58.451543 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:58.454500 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:58.454507 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:58.454513 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:58.454551 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:58.456421 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:58.456937 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:58.456941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:58.456944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:58.457054 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:58.457397 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:58.457403 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:58.457405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:58.457702 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:58.457821 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:58.458356 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 08:38:58.458530 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 08:38:58.459697 4733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:58.443849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:38371 (local address 127.9.174.1:40153)
0807 08:38:58.444555 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:58.444628 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:58.446427 (+ 1799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:58.447042 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:58.447054 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:58.447643 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:58.448414 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:58.448438 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:58.449312 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:58.449317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:58.450793 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:58.450804 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:58.451574 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:58.451577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:58.451726 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:58.452433 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:58.452450 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:58.454310 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:58.456543 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:58.456547 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:58.456560 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:58.456827 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:58.457161 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:58.457163 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:58.457164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:58.457294 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:58.457811 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:58.457815 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:58.458066 (+ 251us) client_negotiation.cc:769] Sending connection context
0807 08:38:58.458281 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 08:38:58.458594 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":193,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
I20260807 08:38:58.461357 4730 heartbeater.cc:347] Connected to a master server at 127.9.174.62:38371
I20260807 08:38:58.461633 4730 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:58.462215 4730 heartbeater.cc:511] Master 127.9.174.62:38371 requested a full tablet report, sending...
I20260807 08:38:58.464233 4466 ts_manager.cc:194] Registered new tserver with Master: d31d813e1cdc476ca4a40d93e7bd6785 (127.9.174.1:43949)
I20260807 08:38:58.465709 4466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:40153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:38:58.482358 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:38371
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:46005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:38:58.614569 4739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:58.614909 4739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:58.614959 4739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:58.619437 4739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:58.619515 4739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:58.619580 4739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:38:58.625247 4739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:46005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:38371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:58.626471 4739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:58.627688 4739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:58.639389 4749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:58.639397 4751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:58.639397 4747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:58.639871 4739 server_base.cc:1034] running on GCE node
I20260807 08:38:58.640353 4739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:58.641139 4739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:58.642338 4739 hybrid_clock.cc:648] HybridClock initialized: now 1786091938642319 us; error 42 us; skew 500 ppm
Aug 07 08:38:58 dist-test-slave-h3ft krb5kdc[4403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091938, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:58.645923 4739 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:38:58.646935 4739 webserver.cc:460] Webserver started at http://127.9.174.2:43901/ using document root <none> and password file <none>
I20260807 08:38:58.647631 4739 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:58.647699 4739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:58.647897 4739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:58.650234 4739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "876a30dcf776423ea61240a07d683120"
format_stamp: "Formatted at 2026-08-07 08:38:58 on dist-test-slave-h3ft"
server_key: "e269c50d9691b4c9d4b41889b34bb0ea"
server_key_iv: "3d5fbee30573b24461d94445a0629ce8"
server_key_version: "encryptionkey@0"
I20260807 08:38:58.650880 4739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "876a30dcf776423ea61240a07d683120"
format_stamp: "Formatted at 2026-08-07 08:38:58 on dist-test-slave-h3ft"
server_key: "e269c50d9691b4c9d4b41889b34bb0ea"
server_key_iv: "3d5fbee30573b24461d94445a0629ce8"
server_key_version: "encryptionkey@0"
I20260807 08:38:58.655251 4739 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 08:38:58.657960 4758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:58.659370 4739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:38:58.659502 4739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "876a30dcf776423ea61240a07d683120"
format_stamp: "Formatted at 2026-08-07 08:38:58 on dist-test-slave-h3ft"
server_key: "e269c50d9691b4c9d4b41889b34bb0ea"
server_key_iv: "3d5fbee30573b24461d94445a0629ce8"
server_key_version: "encryptionkey@0"
I20260807 08:38:58.659626 4739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:58.686944 4739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:58.691643 4739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:58.691902 4739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:58.692713 4739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:58.693833 4739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:58.693887 4739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:58.693976 4739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:58.694016 4739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:58.705111 4739 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:34271
I20260807 08:38:58.705139 4910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:34271 every 8 connection(s)
I20260807 08:38:58.706341 4739 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:38:58.712148 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 4739
I20260807 08:38:58.712320 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:38:58.712735 9912 external_mini_cluster.cc:1428] Setting key c843ef27bcbb9ee3fe9e32a399619ac0
Aug 07 08:38:58 dist-test-slave-h3ft krb5kdc[4403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091938, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:38:58.725585 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:58.709058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:41743 (local address 127.9.174.62:38371)
0807 08:38:58.709268 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:58.709273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:58.710338 (+ 1065us) server_negotiation.cc:407] Connection header received
0807 08:38:58.712073 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:58.712077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:58.712141 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:58.712229 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:58.714074 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:58.714827 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:58.716835 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:58.717073 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:58.717184 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:58.720509 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:58.720517 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:58.720523 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:58.720562 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:58.722536 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:58.723349 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:58.723356 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:58.723360 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:58.723485 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:58.723995 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:58.724000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:58.724005 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:58.724319 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:58.724435 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:58.725163 (+ 728us) server_negotiation.cc:299] Negotiation successful
0807 08:38:58.725333 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
I20260807 08:38:58.726624 4913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:58.709529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:38371 (local address 127.9.174.2:41743)
0807 08:38:58.710143 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:58.710196 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:58.711794 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:58.712480 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:58.712490 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:58.713123 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:58.713887 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:58.713904 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:58.714982 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:58.714985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:58.716686 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:58.716704 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:58.717279 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:58.717285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:58.717558 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:58.718442 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:58.718461 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:58.720231 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:58.722707 (+ 2476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:58.722714 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:58.722733 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:58.723205 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:58.723609 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:58.723612 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:58.723614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:58.723814 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:58.724462 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:58.724469 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:58.724807 (+ 338us) client_negotiation.cc:769] Sending connection context
0807 08:38:58.725041 (+ 234us) client_negotiation.cc:240] Negotiation successful
0807 08:38:58.725403 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":157,"threads_started":1}
I20260807 08:38:58.728276 4911 heartbeater.cc:347] Connected to a master server at 127.9.174.62:38371
I20260807 08:38:58.728611 4911 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
I20260807 08:38:58.729456 4911 heartbeater.cc:511] Master 127.9.174.62:38371 requested a full tablet report, sending...
I20260807 08:38:58.730934 4466 ts_manager.cc:194] Registered new tserver with Master: 876a30dcf776423ea61240a07d683120 (127.9.174.2:34271)
I20260807 08:38:58.731992 4466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:41743
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:38:58.750764 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:38371
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:46005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:38:58.894932 4917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:58.895437 4917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:58.895533 4917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:58.900239 4917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:58.900362 4917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:58.900467 4917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:38:58.907203 4917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:46005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:38371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:58.908933 4917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:58.910466 4917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:58.923062 4930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:58.923069 4929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:58.923069 4935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:58.923492 4917 server_base.cc:1034] running on GCE node
I20260807 08:38:58.924191 4917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:58.925112 4917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:58.926338 4917 hybrid_clock.cc:648] HybridClock initialized: now 1786091938926318 us; error 37 us; skew 500 ppm
Aug 07 08:38:58 dist-test-slave-h3ft krb5kdc[4403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091938, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:58.929978 4917 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:38:58.931330 4917 webserver.cc:460] Webserver started at http://127.9.174.3:37909/ using document root <none> and password file <none>
I20260807 08:38:58.932068 4917 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:58.932142 4917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:58.932371 4917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:58.935250 4917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "c83e2fe233fe4f38b344e1b40276a2f0"
format_stamp: "Formatted at 2026-08-07 08:38:58 on dist-test-slave-h3ft"
server_key: "17c7d885f5e8fa533dacafe159b95283"
server_key_iv: "13b29e03c6e4b22ba51da2d2ddfc7a05"
server_key_version: "encryptionkey@0"
I20260807 08:38:58.936013 4917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "c83e2fe233fe4f38b344e1b40276a2f0"
format_stamp: "Formatted at 2026-08-07 08:38:58 on dist-test-slave-h3ft"
server_key: "17c7d885f5e8fa533dacafe159b95283"
server_key_iv: "13b29e03c6e4b22ba51da2d2ddfc7a05"
server_key_version: "encryptionkey@0"
I20260807 08:38:58.941560 4917 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:38:58.945628 4948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:58.947398 4917 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:38:58.947582 4917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "c83e2fe233fe4f38b344e1b40276a2f0"
format_stamp: "Formatted at 2026-08-07 08:38:58 on dist-test-slave-h3ft"
server_key: "17c7d885f5e8fa533dacafe159b95283"
server_key_iv: "13b29e03c6e4b22ba51da2d2ddfc7a05"
server_key_version: "encryptionkey@0"
I20260807 08:38:58.947746 4917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:58.971215 4917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:58.975391 4917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:58.975704 4917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:58.976526 4917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:58.977627 4917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:58.977710 4917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:58.977797 4917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:58.977850 4917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:58.989347 4917 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:37017
I20260807 08:38:58.989416 5120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:37017 every 8 connection(s)
I20260807 08:38:58.990703 4917 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:38:58.991725 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 4917
I20260807 08:38:58.991892 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:38:58.992233 9912 external_mini_cluster.cc:1428] Setting key 3dedf2afdfc2d079178685cb739378a9
Aug 07 08:38:59 dist-test-slave-h3ft krb5kdc[4403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091938, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:38:59.009023 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:58.993658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:38453 (local address 127.9.174.62:38371)
0807 08:38:58.993927 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:58.993932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:58.995334 (+ 1402us) server_negotiation.cc:407] Connection header received
0807 08:38:58.997041 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:58.997045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:58.997125 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:58.997239 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:58.998753 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:58.999425 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.000806 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.001053 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.001182 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.004461 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:59.004465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:59.004467 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:59.004499 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:59.006367 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:59.007046 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:59.007055 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:59.007058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:59.007168 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:59.007606 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:59.007611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:59.007613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:59.007861 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:59.007982 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:59.008627 (+ 645us) server_negotiation.cc:299] Negotiation successful
0807 08:38:59.008786 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260807 08:38:59.010237 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:58.994285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:38371 (local address 127.9.174.3:38453)
0807 08:38:58.995124 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:58.995202 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:58.996762 (+ 1560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:58.997469 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:58.997477 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:58.997942 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:58.998585 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:58.998601 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:58.999585 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:58.999591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.000655 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.000667 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.001252 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.001256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.001467 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.002275 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:59.002295 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:59.004218 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:59.006526 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:59.006533 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:59.006558 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:59.006892 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:59.007276 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:59.007280 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:59.007282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:59.007496 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:59.007983 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:59.007994 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:59.008336 (+ 342us) client_negotiation.cc:769] Sending connection context
0807 08:38:59.008583 (+ 247us) client_negotiation.cc:240] Negotiation successful
0807 08:38:59.009055 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":226,"threads_started":1}
I20260807 08:38:59.011730 5121 heartbeater.cc:347] Connected to a master server at 127.9.174.62:38371
I20260807 08:38:59.012089 5121 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:59.012774 5121 heartbeater.cc:511] Master 127.9.174.62:38371 requested a full tablet report, sending...
I20260807 08:38:59.014189 4466 ts_manager.cc:194] Registered new tserver with Master: c83e2fe233fe4f38b344e1b40276a2f0 (127.9.174.3:37017)
I20260807 08:38:59.014958 4466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:38453
I20260807 08:38:59.019801 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:38:59.031642 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.022449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45914 (local address 127.9.174.62:38371)
0807 08:38:59.022618 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:59.022623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:59.022837 (+ 214us) server_negotiation.cc:407] Connection header received
0807 08:38:59.023036 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:59.023040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:59.023105 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:59.023196 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:59.024021 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.024604 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.025544 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.025803 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.025921 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.027904 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:59.027908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:59.027911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:59.027950 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:59.029441 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:59.030067 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:59.030071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:59.030073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:59.030171 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:59.030538 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:59.030542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:59.030544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:59.030820 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:59.031003 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:59.031319 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 08:38:59.031434 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 08:38:59.036031 4466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:38:59.040131 4466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:38:59.059743 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.051871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:34271 (local address 127.0.0.1:46560)
0807 08:38:59.053053 (+ 1182us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:59.053121 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:59.053294 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:59.054273 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:59.054280 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:59.054308 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:59.054525 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.054533 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.056182 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.056188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.057857 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.057869 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.058976 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.058980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.059168 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.059227 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 08:38:59.059335 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 08:38:59.059527 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":958,"mutex_wait_us":29,"thread_start_us":196,"threads_started":1}
I20260807 08:38:59.059911 5143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.053399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47116 (local address 127.9.174.3:37017)
0807 08:38:59.053890 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:59.053908 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:59.054375 (+ 467us) server_negotiation.cc:407] Connection header received
0807 08:38:59.054469 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:59.054473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:59.054652 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:59.054792 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:59.055311 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.056510 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.057826 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.058630 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.058750 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.058963 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:59.059601 (+ 638us) server_negotiation.cc:299] Negotiation successful
0807 08:38:59.059734 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":135,"threads_started":1}
I20260807 08:38:59.060384 5136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.052115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46560 (local address 127.9.174.2:34271)
0807 08:38:59.052648 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:59.052661 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:59.053273 (+ 612us) server_negotiation.cc:407] Connection header received
0807 08:38:59.053872 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:59.053878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:59.054092 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:59.054283 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:59.054708 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.056005 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.058043 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.058812 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.059834 (+ 1022us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.060025 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:59.060089 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 08:38:59.060204 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":158,"threads_started":1}
I20260807 08:38:59.060429 5146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.053741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59138 (local address 127.9.174.1:43949)
0807 08:38:59.054130 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:59.054139 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:59.054170 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:38:59.054261 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:59.054266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:59.054489 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:59.054642 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:59.054915 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.056507 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.058252 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.059154 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.059885 (+ 731us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.060067 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:59.060133 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 08:38:59.060243 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":174,"threads_started":1}
I20260807 08:38:59.060446 5137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.053230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:37017 (local address 127.0.0.1:47116)
0807 08:38:59.053597 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:59.054105 (+ 508us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:59.054258 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:59.055031 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:59.055036 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:59.055060 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:59.055161 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.055167 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.056697 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.056699 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.057675 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.057685 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.059341 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.059344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.059427 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.059448 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 08:38:59.060225 (+ 777us) client_negotiation.cc:240] Negotiation successful
0807 08:38:59.060316 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":67,"thread_start_us":110,"threads_started":1}
I20260807 08:38:59.059743 5140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.052856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:43949 (local address 127.0.0.1:59138)
0807 08:38:59.053521 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:59.053541 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:59.053654 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:59.054637 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:59.054640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:59.054661 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:59.054766 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.054775 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.056650 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.056653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.058106 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.058115 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.059292 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.059296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.059384 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.059406 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:38:59.059470 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:38:59.059554 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":71,"thread_start_us":129,"threads_started":1}
I20260807 08:38:59.064446 4826 tablet_service.cc:1467] Processing CreateTablet for tablet 8813aa5f01f74a50a6cf5fb5b4668dbf (DEFAULT_TABLE table=test-table [id=0ac483ca30684923af4f6f0d4350cbc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:38:59.064445 5017 tablet_service.cc:1467] Processing CreateTablet for tablet 8813aa5f01f74a50a6cf5fb5b4668dbf (DEFAULT_TABLE table=test-table [id=0ac483ca30684923af4f6f0d4350cbc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:38:59.064678 4646 tablet_service.cc:1467] Processing CreateTablet for tablet 8813aa5f01f74a50a6cf5fb5b4668dbf (DEFAULT_TABLE table=test-table [id=0ac483ca30684923af4f6f0d4350cbc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:38:59.065769 4826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8813aa5f01f74a50a6cf5fb5b4668dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:59.065771 5017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8813aa5f01f74a50a6cf5fb5b4668dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:59.065773 4646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8813aa5f01f74a50a6cf5fb5b4668dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:59.075553 5147 tablet_bootstrap.cc:492] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785: Bootstrap starting.
I20260807 08:38:59.076417 5148 tablet_bootstrap.cc:492] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0: Bootstrap starting.
I20260807 08:38:59.078964 5148 tablet_bootstrap.cc:654] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:59.078960 5147 tablet_bootstrap.cc:654] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:59.079475 5150 tablet_bootstrap.cc:492] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120: Bootstrap starting.
I20260807 08:38:59.080257 5147 log.cc:824] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:59.080520 5148 log.cc:824] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:59.082459 5150 tablet_bootstrap.cc:654] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:59.083217 5147 tablet_bootstrap.cc:492] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785: No bootstrap required, opened a new log
I20260807 08:38:59.083498 5147 ts_tablet_manager.cc:1397] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 08:38:59.083560 5148 tablet_bootstrap.cc:492] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0: No bootstrap required, opened a new log
I20260807 08:38:59.083537 5150 log.cc:824] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:59.083741 5148 ts_tablet_manager.cc:1397] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 08:38:59.085404 5150 tablet_bootstrap.cc:492] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120: No bootstrap required, opened a new log
I20260807 08:38:59.085639 5150 ts_tablet_manager.cc:1397] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 08:38:59.088430 5148 raft_consensus.cc:348] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876a30dcf776423ea61240a07d683120" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 34271 } } peers { permanent_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43949 } } peers { permanent_uuid: "c83e2fe233fe4f38b344e1b40276a2f0" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37017 } }
I20260807 08:38:59.088845 5148 raft_consensus.cc:374] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:59.088745 5147 raft_consensus.cc:348] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876a30dcf776423ea61240a07d683120" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 34271 } } peers { permanent_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43949 } } peers { permanent_uuid: "c83e2fe233fe4f38b344e1b40276a2f0" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37017 } }
I20260807 08:38:59.088966 5148 raft_consensus.cc:729] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c83e2fe233fe4f38b344e1b40276a2f0, State: Initialized, Role: FOLLOWER
I20260807 08:38:59.089075 5147 raft_consensus.cc:374] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:59.088867 5150 raft_consensus.cc:348] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876a30dcf776423ea61240a07d683120" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 34271 } } peers { permanent_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43949 } } peers { permanent_uuid: "c83e2fe233fe4f38b344e1b40276a2f0" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37017 } }
I20260807 08:38:59.089141 5147 raft_consensus.cc:729] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d31d813e1cdc476ca4a40d93e7bd6785, State: Initialized, Role: FOLLOWER
I20260807 08:38:59.089192 5150 raft_consensus.cc:374] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:59.089262 5150 raft_consensus.cc:729] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 876a30dcf776423ea61240a07d683120, State: Initialized, Role: FOLLOWER
I20260807 08:38:59.089666 5148 consensus_queue.cc:260] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876a30dcf776423ea61240a07d683120" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 34271 } } peers { permanent_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43949 } } peers { permanent_uuid: "c83e2fe233fe4f38b344e1b40276a2f0" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37017 } }
I20260807 08:38:59.089666 5147 consensus_queue.cc:260] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876a30dcf776423ea61240a07d683120" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 34271 } } peers { permanent_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43949 } } peers { permanent_uuid: "c83e2fe233fe4f38b344e1b40276a2f0" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37017 } }
I20260807 08:38:59.089761 5150 consensus_queue.cc:260] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876a30dcf776423ea61240a07d683120" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 34271 } } peers { permanent_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43949 } } peers { permanent_uuid: "c83e2fe233fe4f38b344e1b40276a2f0" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37017 } }
I20260807 08:38:59.090628 4730 heartbeater.cc:503] Master 127.9.174.62:38371 was elected leader, sending a full tablet report...
I20260807 08:38:59.091094 5148 ts_tablet_manager.cc:1428] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 08:38:59.091094 5147 ts_tablet_manager.cc:1428] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785: Time spent starting tablet: real 0.008s user 0.003s sys 0.005s
I20260807 08:38:59.091229 5121 heartbeater.cc:503] Master 127.9.174.62:38371 was elected leader, sending a full tablet report...
I20260807 08:38:59.092511 4911 heartbeater.cc:503] Master 127.9.174.62:38371 was elected leader, sending a full tablet report...
I20260807 08:38:59.093879 5150 ts_tablet_manager.cc:1428] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 08:38:59.116106 5155 raft_consensus.cc:482] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:38:59.116556 5155 raft_consensus.cc:504] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876a30dcf776423ea61240a07d683120" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 34271 } } peers { permanent_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43949 } } peers { permanent_uuid: "c83e2fe233fe4f38b344e1b40276a2f0" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37017 } }
I20260807 08:38:59.118257 5155 leader_election.cc:288] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 876a30dcf776423ea61240a07d683120 (127.9.174.2:34271), c83e2fe233fe4f38b344e1b40276a2f0 (127.9.174.3:37017)
I20260807 08:38:59.120136 5159 raft_consensus.cc:482] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:38:59.120560 5159 raft_consensus.cc:504] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876a30dcf776423ea61240a07d683120" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 34271 } } peers { permanent_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43949 } } peers { permanent_uuid: "c83e2fe233fe4f38b344e1b40276a2f0" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37017 } }
I20260807 08:38:59.123306 5159 leader_election.cc:288] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d31d813e1cdc476ca4a40d93e7bd6785 (127.9.174.1:43949), c83e2fe233fe4f38b344e1b40276a2f0 (127.9.174.3:37017)
I20260807 08:38:59.124118 5143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.119557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:48373 (local address 127.9.174.3:37017)
0807 08:38:59.119711 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:59.119716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:59.119732 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:38:59.119838 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:59.119841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:59.119889 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:59.119965 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:59.120384 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.121152 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.122649 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.123344 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.123428 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.123519 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:59.123863 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 08:38:59.123951 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:38:59.124562 5160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.118534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:34271 (local address 127.9.174.1:57893)
0807 08:38:59.119050 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:59.119072 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:59.119203 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:59.119687 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:59.119692 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:59.119738 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:59.119945 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.119953 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.121071 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.121074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.122406 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.122462 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.123774 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.123777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.123910 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.123943 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 08:38:59.124037 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 08:38:59.124119 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"spinlock_wait_cycles":33792,"thread_start_us":212,"threads_started":1}
I20260807 08:38:59.125010 5136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.118959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:57893 (local address 127.9.174.2:34271)
0807 08:38:59.119214 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:59.119220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:59.119239 (+ 19us) server_negotiation.cc:407] Connection header received
0807 08:38:59.119309 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:59.119313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:59.119371 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:59.119453 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:59.120114 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.120906 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.122823 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.123500 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.123589 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.123654 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:59.124323 (+ 669us) server_negotiation.cc:299] Negotiation successful
0807 08:38:59.124856 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":417,"server-negotiator.queue_time_us":159}
I20260807 08:38:59.124749 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.119163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:37017 (local address 127.9.174.1:48373)
0807 08:38:59.119605 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:59.119621 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:59.119745 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:59.120105 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:59.120109 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:59.120129 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:59.120255 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.120260 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.121303 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.121307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.122490 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.122502 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.123615 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.123618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.123717 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.123742 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 08:38:59.124471 (+ 729us) client_negotiation.cc:240] Negotiation successful
0807 08:38:59.124548 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":1,"spinlock_wait_cycles":13184,"thread_start_us":115,"threads_started":1}
I20260807 08:38:59.125999 4855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8813aa5f01f74a50a6cf5fb5b4668dbf" candidate_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "876a30dcf776423ea61240a07d683120" is_pre_election: true
I20260807 08:38:59.126267 4855 raft_consensus.cc:2454] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d31d813e1cdc476ca4a40d93e7bd6785 in term 0.
I20260807 08:38:59.126232 5047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8813aa5f01f74a50a6cf5fb5b4668dbf" candidate_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c83e2fe233fe4f38b344e1b40276a2f0" is_pre_election: true
I20260807 08:38:59.126567 5047 raft_consensus.cc:2454] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d31d813e1cdc476ca4a40d93e7bd6785 in term 0.
I20260807 08:38:59.126814 4913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.122595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:43949 (local address 127.9.174.2:34875)
0807 08:38:59.122731 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:59.122747 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:59.122840 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:59.123191 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:59.123194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:59.123215 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:59.123334 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.123340 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.124285 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.124289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.125476 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.125499 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.126414 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.126418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.126538 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.126561 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:38:59.126630 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 08:38:59.126692 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 08:38:59.127166 5146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.122573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:34875 (local address 127.9.174.1:43949)
0807 08:38:59.122690 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:59.122694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:59.122923 (+ 229us) server_negotiation.cc:407] Connection header received
0807 08:38:59.122964 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:59.122967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:59.123031 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:59.123094 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:59.123466 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.124146 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.125629 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.126193 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.126288 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.126351 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:59.126971 (+ 620us) server_negotiation.cc:299] Negotiation successful
0807 08:38:59.127062 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:38:59.127254 4577 leader_election.cc:302] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 876a30dcf776423ea61240a07d683120, d31d813e1cdc476ca4a40d93e7bd6785; no voters:
I20260807 08:38:59.127578 5155 raft_consensus.cc:2781] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:38:59.127646 5155 raft_consensus.cc:482] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:38:59.127683 5155 raft_consensus.cc:3037] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:59.127679 4667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8813aa5f01f74a50a6cf5fb5b4668dbf" candidate_uuid: "876a30dcf776423ea61240a07d683120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" is_pre_election: true
I20260807 08:38:59.128727 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.123672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:41121 (local address 127.9.174.3:37017)
0807 08:38:59.124097 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:59.124112 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:59.124364 (+ 252us) server_negotiation.cc:407] Connection header received
0807 08:38:59.124559 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:59.124563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:59.124625 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:59.124699 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:59.125096 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.125857 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.127308 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.128011 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.128105 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.128169 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:59.128499 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 08:38:59.128589 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":146,"threads_started":1}
I20260807 08:38:59.129227 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.123034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:37017 (local address 127.9.174.2:41121)
0807 08:38:59.124280 (+ 1246us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:59.124295 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:59.124427 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:59.124835 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:59.124837 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:59.124854 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:59.124967 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.124972 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.125977 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.125981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.127160 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.127172 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.128242 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.128246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.128354 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.128383 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 08:38:59.128982 (+ 599us) client_negotiation.cc:240] Negotiation successful
0807 08:38:59.129080 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1172,"thread_start_us":141,"threads_started":1}
I20260807 08:38:59.129325 5155 raft_consensus.cc:504] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876a30dcf776423ea61240a07d683120" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 34271 } } peers { permanent_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43949 } } peers { permanent_uuid: "c83e2fe233fe4f38b344e1b40276a2f0" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37017 } }
I20260807 08:38:59.129628 4667 raft_consensus.cc:2379] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 876a30dcf776423ea61240a07d683120 in current term 1: Already voted for candidate d31d813e1cdc476ca4a40d93e7bd6785 in this term.
I20260807 08:38:59.129666 5047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8813aa5f01f74a50a6cf5fb5b4668dbf" candidate_uuid: "876a30dcf776423ea61240a07d683120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c83e2fe233fe4f38b344e1b40276a2f0" is_pre_election: true
I20260807 08:38:59.129706 5155 leader_election.cc:288] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [CANDIDATE]: Term 1 election: Requested vote from peers 876a30dcf776423ea61240a07d683120 (127.9.174.2:34271), c83e2fe233fe4f38b344e1b40276a2f0 (127.9.174.3:37017)
I20260807 08:38:59.129788 5047 raft_consensus.cc:2454] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 876a30dcf776423ea61240a07d683120 in term 0.
I20260807 08:38:59.130191 4855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8813aa5f01f74a50a6cf5fb5b4668dbf" candidate_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "876a30dcf776423ea61240a07d683120"
I20260807 08:38:59.130262 4759 leader_election.cc:302] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 876a30dcf776423ea61240a07d683120, c83e2fe233fe4f38b344e1b40276a2f0; no voters:
I20260807 08:38:59.130311 4855 raft_consensus.cc:3037] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:59.131129 5047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8813aa5f01f74a50a6cf5fb5b4668dbf" candidate_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c83e2fe233fe4f38b344e1b40276a2f0"
I20260807 08:38:59.131285 5047 raft_consensus.cc:3037] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:59.131798 4855 raft_consensus.cc:2454] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d31d813e1cdc476ca4a40d93e7bd6785 in term 1.
I20260807 08:38:59.131990 5159 raft_consensus.cc:2741] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260807 08:38:59.132184 4577 leader_election.cc:302] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 876a30dcf776423ea61240a07d683120, d31d813e1cdc476ca4a40d93e7bd6785; no voters:
I20260807 08:38:59.132546 5155 raft_consensus.cc:2781] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:38:59.132815 5155 raft_consensus.cc:686] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [term 1 LEADER]: Becoming Leader. State: Replica: d31d813e1cdc476ca4a40d93e7bd6785, State: Running, Role: LEADER
I20260807 08:38:59.132954 5047 raft_consensus.cc:2454] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d31d813e1cdc476ca4a40d93e7bd6785 in term 1.
I20260807 08:38:59.133205 5155 consensus_queue.cc:237] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876a30dcf776423ea61240a07d683120" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 34271 } } peers { permanent_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43949 } } peers { permanent_uuid: "c83e2fe233fe4f38b344e1b40276a2f0" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37017 } }
I20260807 08:38:59.137590 4466 catalog_manager.cc:5495] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 reported cstate change: term changed from 0 to 1, leader changed from <none> to d31d813e1cdc476ca4a40d93e7bd6785 (127.9.174.1). New cstate: current_term: 1 leader_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876a30dcf776423ea61240a07d683120" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 34271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d31d813e1cdc476ca4a40d93e7bd6785" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43949 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c83e2fe233fe4f38b344e1b40276a2f0" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37017 } health_report { overall_health: UNKNOWN } } }
I20260807 08:38:59.163494 5146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.159595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59148 (local address 127.9.174.1:43949)
0807 08:38:59.159760 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:59.159764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:59.159779 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:38:59.159875 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:59.159877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:59.159946 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:59.160082 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:38:59.160336 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.160888 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.161752 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.161971 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.162071 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.162364 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:38:59.162879 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:38:59.163056 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:59.163228 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 08:38:59.163286 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 08:38:59.171277 4855 raft_consensus.cc:1261] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120 [term 1 FOLLOWER]: Refusing update from remote peer d31d813e1cdc476ca4a40d93e7bd6785: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:38:59.171277 5047 raft_consensus.cc:1261] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0 [term 1 FOLLOWER]: Refusing update from remote peer d31d813e1cdc476ca4a40d93e7bd6785: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:38:59.172102 5155 consensus_queue.cc:1035] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "876a30dcf776423ea61240a07d683120" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 34271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:38:59.172372 5167 consensus_queue.cc:1035] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c83e2fe233fe4f38b344e1b40276a2f0" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 37017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:38:59.182303 5171 mvcc.cc:205] Tried to move back new op lower bound from 7315832582835761152 to 7315832582693150720. Current Snapshot: MvccSnapshot[applied={T|T < 7315832582835761152}]
I20260807 08:38:59.194263 4464 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:38:59.194566 4464 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:38:59.198133 4464 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0a22be852cb452289a73251be101e6d: Sending DeleteTablet for 3 replicas of tablet 8813aa5f01f74a50a6cf5fb5b4668dbf
I20260807 08:38:59.199321 4826 tablet_service.cc:1514] Processing DeleteTablet for tablet 8813aa5f01f74a50a6cf5fb5b4668dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:38:59 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:46560
I20260807 08:38:59.199445 4646 tablet_service.cc:1514] Processing DeleteTablet for tablet 8813aa5f01f74a50a6cf5fb5b4668dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:38:59 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:59138
I20260807 08:38:59.199631 5017 tablet_service.cc:1514] Processing DeleteTablet for tablet 8813aa5f01f74a50a6cf5fb5b4668dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:38:59 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:47116
I20260807 08:38:59.200356 5181 tablet_replica.cc:330] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120: stopping tablet replica
I20260807 08:38:59.200383 5183 tablet_replica.cc:330] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785: stopping tablet replica
I20260807 08:38:59.200824 5181 raft_consensus.cc:2229] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:38:59.200829 5183 raft_consensus.cc:2229] T 8813aa5f01f74a50a6cf5fb5b4668dbf P d31d813e1cdc476ca4a40d93e7bd6785 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:38:59.200892 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 4556
I20260807 08:38:59.204525 5181 raft_consensus.cc:2258] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:38:59.209910 5184 tablet_replica.cc:330] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0: stopping tablet replica
I20260807 08:38:59.210542 5184 raft_consensus.cc:2229] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:38:59.211109 5184 raft_consensus.cc:2258] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:38:59.212621 5181 ts_tablet_manager.cc:1905] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:38:59.212730 5184 ts_tablet_manager.cc:1905] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:38:59.216401 5184 ts_tablet_manager.cc:1918] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:38:59.216542 5184 log.cc:1196] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/8813aa5f01f74a50a6cf5fb5b4668dbf
I20260807 08:38:59.216547 5181 ts_tablet_manager.cc:1918] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:38:59.216653 5181 log.cc:1196] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/8813aa5f01f74a50a6cf5fb5b4668dbf
I20260807 08:38:59.217031 5181 ts_tablet_manager.cc:1939] T 8813aa5f01f74a50a6cf5fb5b4668dbf P 876a30dcf776423ea61240a07d683120: Deleting consensus metadata
I20260807 08:38:59.217042 5184 ts_tablet_manager.cc:1939] T 8813aa5f01f74a50a6cf5fb5b4668dbf P c83e2fe233fe4f38b344e1b40276a2f0: Deleting consensus metadata
I20260807 08:38:59.218806 4447 catalog_manager.cc:4841] TS c83e2fe233fe4f38b344e1b40276a2f0 (127.9.174.3:37017): tablet 8813aa5f01f74a50a6cf5fb5b4668dbf (table test-table [id=0ac483ca30684923af4f6f0d4350cbc5]) successfully deleted
I20260807 08:38:59.219192 4447 catalog_manager.cc:4841] TS 876a30dcf776423ea61240a07d683120 (127.9.174.2:34271): tablet 8813aa5f01f74a50a6cf5fb5b4668dbf (table test-table [id=0ac483ca30684923af4f6f0d4350cbc5]) successfully deleted
W20260807 08:38:59.219453 4447 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:43949 (error 108)
I20260807 08:38:59.219897 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 4739
I20260807 08:38:59.226040 5140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.225503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:43949 (local address 127.0.0.1:59154)
0807 08:38:59.225745 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:59.225883 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:43949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260807 08:38:59.226372 4447 catalog_manager.cc:4584] TS d31d813e1cdc476ca4a40d93e7bd6785 (127.9.174.1:43949): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8813aa5f01f74a50a6cf5fb5b4668dbf: Network error: Client connection negotiation failed: client connection to 127.9.174.1:43949: connect: Connection refused (error 111)
I20260807 08:38:59.237843 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 4917
I20260807 08:38:59.255100 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 4419
2026-08-07T08:38:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2785 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T08:38:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:38:59Z Disabled control of system clock
I20260807 08:38:59.305151 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:43681
--webserver_interface=127.9.174.62
--webserver_port=0
--builtin_ntp_servers=127.9.174.20:44617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:43681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:38:59.439675 5194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:59.440019 5194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:59.440059 5194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:59.444167 5194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:38:59.444231 5194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:59.444247 5194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:59.444263 5194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:38:59.444278 5194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:38:59.449136 5194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:44617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:43681
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:43681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:59.450508 5194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:59.452353 5194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:59.463075 5203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:59.463075 5202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:59.463152 5194 server_base.cc:1034] running on GCE node
W20260807 08:38:59.463075 5206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:59.464258 5194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:59.465771 5194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:59.466993 5194 hybrid_clock.cc:648] HybridClock initialized: now 1786091939466962 us; error 50 us; skew 500 ppm
I20260807 08:38:59.472966 5194 webserver.cc:460] Webserver started at http://127.9.174.62:35037/ using document root <none> and password file <none>
I20260807 08:38:59.473839 5194 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:59.473956 5194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:59.474278 5194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:59.477166 5194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "6571789447dd4a4e93c91b26373aef37"
format_stamp: "Formatted at 2026-08-07 08:38:59 on dist-test-slave-h3ft"
server_key: "7175afc487903238f0ca4e72b995e4f3"
server_key_iv: "62e4cc4d56734434d8df577aee9132ad"
server_key_version: "encryptionkey@0"
I20260807 08:38:59.477960 5194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "6571789447dd4a4e93c91b26373aef37"
format_stamp: "Formatted at 2026-08-07 08:38:59 on dist-test-slave-h3ft"
server_key: "7175afc487903238f0ca4e72b995e4f3"
server_key_iv: "62e4cc4d56734434d8df577aee9132ad"
server_key_version: "encryptionkey@0"
I20260807 08:38:59.482868 5194 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 08:38:59.485996 5215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:59.487676 5194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:38:59.487851 5194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "6571789447dd4a4e93c91b26373aef37"
format_stamp: "Formatted at 2026-08-07 08:38:59 on dist-test-slave-h3ft"
server_key: "7175afc487903238f0ca4e72b995e4f3"
server_key_iv: "62e4cc4d56734434d8df577aee9132ad"
server_key_version: "encryptionkey@0"
I20260807 08:38:59.487998 5194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:59.511215 5194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:59.512260 5194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:59.512518 5194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:59.521543 5194 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:43681
I20260807 08:38:59.521553 5287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:43681 every 8 connection(s)
I20260807 08:38:59.522735 5194 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:38:59.523695 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 5194
I20260807 08:38:59.523878 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:38:59.524328 9912 external_mini_cluster.cc:1428] Setting key 5b5f85eeadba1812dae0645893bfced9
I20260807 08:38:59.527382 5288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:59.534030 5288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37: Bootstrap starting.
I20260807 08:38:59.534833 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.526061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44496 (local address 127.9.174.62:43681)
0807 08:38:59.526823 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:59.526841 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:59.526885 (+ 44us) server_negotiation.cc:407] Connection header received
0807 08:38:59.527769 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:59.527790 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:59.528242 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:59.528613 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:59.529105 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.530155 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.531140 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.531649 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.531833 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.532851 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:59.532856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:59.532870 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:38:59.533317 (+ 447us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:59.533379 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:38:59.533400 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:59.533514 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:59.533644 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 08:38:59.533940 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":183,"threads_started":1}
I20260807 08:38:59.537003 5288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:59.538013 5288 log.cc:824] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:59.540282 5288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37: No bootstrap required, opened a new log
I20260807 08:38:59.544612 5288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6571789447dd4a4e93c91b26373aef37" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43681 } }
I20260807 08:38:59.544888 5288 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:59.544925 5288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6571789447dd4a4e93c91b26373aef37, State: Initialized, Role: FOLLOWER
I20260807 08:38:59.545415 5288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6571789447dd4a4e93c91b26373aef37" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43681 } }
I20260807 08:38:59.545565 5288 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:38:59.545617 5288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:38:59.545698 5288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:59.546770 5288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6571789447dd4a4e93c91b26373aef37" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43681 } }
I20260807 08:38:59.547216 5288 leader_election.cc:302] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6571789447dd4a4e93c91b26373aef37; no voters:
I20260807 08:38:59.547544 5288 leader_election.cc:288] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:38:59.547689 5296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:38:59.547967 5296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [term 1 LEADER]: Becoming Leader. State: Replica: 6571789447dd4a4e93c91b26373aef37, State: Running, Role: LEADER
I20260807 08:38:59.548418 5296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6571789447dd4a4e93c91b26373aef37" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43681 } }
I20260807 08:38:59.549207 5288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:38:59.550441 5297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6571789447dd4a4e93c91b26373aef37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6571789447dd4a4e93c91b26373aef37" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43681 } } }
I20260807 08:38:59.550441 5298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6571789447dd4a4e93c91b26373aef37. Latest consensus state: current_term: 1 leader_uuid: "6571789447dd4a4e93c91b26373aef37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6571789447dd4a4e93c91b26373aef37" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 43681 } } }
I20260807 08:38:59.550624 5297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:59.550637 5298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37 [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:59.551120 5307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:38:59.554566 5307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:38:59.561270 5307 catalog_manager.cc:1347] Generated new cluster ID: d847b7392c1a491496d50972f880975b
I20260807 08:38:59.561364 5307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:38:59.584992 5307 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:38:59.586364 5307 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:38:59.597472 5307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37: Generated new TSK 0
I20260807 08:38:59.598706 5307 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:38:59.604382 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43681
--builtin_ntp_servers=127.9.174.20:44617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:38:59.779472 5320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:59.779842 5320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:59.779920 5320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:59.784230 5320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:59.784327 5320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:59.784392 5320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:38:59.789601 5320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:44617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:38:59.791143 5320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:59.793257 5320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:59.805171 5328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:59.805203 5332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:59.805248 5329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:59.805594 5320 server_base.cc:1034] running on GCE node
I20260807 08:38:59.806082 5320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:59.806946 5320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:59.808188 5320 hybrid_clock.cc:648] HybridClock initialized: now 1786091939808150 us; error 47 us; skew 500 ppm
I20260807 08:38:59.815542 5320 webserver.cc:460] Webserver started at http://127.9.174.1:39793/ using document root <none> and password file <none>
I20260807 08:38:59.816371 5320 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:59.816501 5320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:59.816802 5320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:59.819540 5320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "f5012659eacb4a51b1851ff2997867c9"
format_stamp: "Formatted at 2026-08-07 08:38:59 on dist-test-slave-h3ft"
server_key: "66f521d8dbbae91a578d93b7ded28011"
server_key_iv: "495960231e44f8b7c267afb693a4805d"
server_key_version: "encryptionkey@0"
I20260807 08:38:59.820271 5320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "f5012659eacb4a51b1851ff2997867c9"
format_stamp: "Formatted at 2026-08-07 08:38:59 on dist-test-slave-h3ft"
server_key: "66f521d8dbbae91a578d93b7ded28011"
server_key_iv: "495960231e44f8b7c267afb693a4805d"
server_key_version: "encryptionkey@0"
I20260807 08:38:59.824790 5320 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 08:38:59.827970 5342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:59.829608 5320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:38:59.829807 5320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "f5012659eacb4a51b1851ff2997867c9"
format_stamp: "Formatted at 2026-08-07 08:38:59 on dist-test-slave-h3ft"
server_key: "66f521d8dbbae91a578d93b7ded28011"
server_key_iv: "495960231e44f8b7c267afb693a4805d"
server_key_version: "encryptionkey@0"
I20260807 08:38:59.829977 5320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:59.851747 5320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:59.852721 5320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:59.852928 5320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:59.853638 5320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:59.854820 5320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:59.854872 5320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:59.854923 5320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:59.854944 5320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:59.866217 5320 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:36139
I20260807 08:38:59.866243 5518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:36139 every 8 connection(s)
I20260807 08:38:59.867559 5320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:38:59.868938 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 5320
I20260807 08:38:59.869113 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:38:59.869483 9912 external_mini_cluster.cc:1428] Setting key 4cdf0bf2f190c3307da7b99df4f8aa3b
I20260807 08:38:59.872254 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43681
--builtin_ntp_servers=127.9.174.20:44617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:38:59.880683 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.870080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:33449 (local address 127.9.174.62:43681)
0807 08:38:59.870260 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:59.870263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:59.871874 (+ 1611us) server_negotiation.cc:407] Connection header received
0807 08:38:59.873702 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:59.873709 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:59.873790 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:59.873892 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:59.875055 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.875671 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.877603 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.877844 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.877932 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.880043 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:59.880046 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:59.880049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:38:59.880066 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:59.880090 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:38:59.880099 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:59.880164 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:59.880414 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 08:38:59.880523 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 08:38:59.882319 5522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:59.870675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43681 (local address 127.9.174.1:33449)
0807 08:38:59.871705 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:59.871760 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:59.873394 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:59.874094 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:59.874103 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:59.874205 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:59.874848 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.874862 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.875833 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.875843 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.877431 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:59.877449 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:59.878040 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:59.878047 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:59.878347 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:59.879621 (+ 1274us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:38:59.879624 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:59.879704 (+ 80us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:38:59.879832 (+ 128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:59.880197 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:59.880211 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:59.880223 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 08:38:59.880415 (+ 192us) client_negotiation.cc:240] Negotiation successful
0807 08:38:59.880855 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":140,"threads_started":1}
I20260807 08:38:59.884037 5519 heartbeater.cc:347] Connected to a master server at 127.9.174.62:43681
I20260807 08:38:59.884418 5519 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:59.885371 5519 heartbeater.cc:511] Master 127.9.174.62:43681 requested a full tablet report, sending...
I20260807 08:38:59.887632 5241 ts_manager.cc:194] Registered new tserver with Master: f5012659eacb4a51b1851ff2997867c9 (127.9.174.1:36139)
I20260807 08:38:59.889129 5241 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.174.1:33449
W20260807 08:39:00.021030 5523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:00.021488 5523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:00.021569 5523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:00.026275 5523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:00.026396 5523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:00.026499 5523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:39:00.031944 5523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:44617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:00.033525 5523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:00.035465 5523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:00.048346 5532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:00.048336 5528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:00.048317 5529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:00.048774 5523 server_base.cc:1034] running on GCE node
I20260807 08:39:00.049316 5523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:00.050324 5523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:00.051596 5523 hybrid_clock.cc:648] HybridClock initialized: now 1786091940051567 us; error 35 us; skew 500 ppm
I20260807 08:39:00.054276 5523 webserver.cc:460] Webserver started at http://127.9.174.2:37889/ using document root <none> and password file <none>
I20260807 08:39:00.055143 5523 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:00.055228 5523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:00.055540 5523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:00.058292 5523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "8cf5bdd38a5245b486800d3de10c52ec"
format_stamp: "Formatted at 2026-08-07 08:39:00 on dist-test-slave-h3ft"
server_key: "1f1fcf8b47213db5156e3accd2fbd2cc"
server_key_iv: "b366ea7e4193d77d46071d7689f24bdc"
server_key_version: "encryptionkey@0"
I20260807 08:39:00.059116 5523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "8cf5bdd38a5245b486800d3de10c52ec"
format_stamp: "Formatted at 2026-08-07 08:39:00 on dist-test-slave-h3ft"
server_key: "1f1fcf8b47213db5156e3accd2fbd2cc"
server_key_iv: "b366ea7e4193d77d46071d7689f24bdc"
server_key_version: "encryptionkey@0"
I20260807 08:39:00.064157 5523 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 08:39:00.067520 5543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:00.069096 5523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:39:00.069255 5523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "8cf5bdd38a5245b486800d3de10c52ec"
format_stamp: "Formatted at 2026-08-07 08:39:00 on dist-test-slave-h3ft"
server_key: "1f1fcf8b47213db5156e3accd2fbd2cc"
server_key_iv: "b366ea7e4193d77d46071d7689f24bdc"
server_key_version: "encryptionkey@0"
I20260807 08:39:00.069399 5523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:00.092029 5523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:00.093252 5523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:00.093530 5523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:00.094362 5523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:00.095686 5523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:00.095762 5523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:00.095863 5523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:00.095912 5523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:00.107928 5523 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:44127
I20260807 08:39:00.107949 5697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:44127 every 8 connection(s)
I20260807 08:39:00.109134 5523 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:39:00.114399 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 5523
I20260807 08:39:00.114567 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:39:00.114959 9912 external_mini_cluster.cc:1428] Setting key 3535e5a16d0b179f3f4410e6f8d1f8e6
I20260807 08:39:00.117575 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43681
--builtin_ntp_servers=127.9.174.20:44617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:39:00.122063 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.111685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:46813 (local address 127.9.174.62:43681)
0807 08:39:00.111895 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:00.111899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:00.113321 (+ 1422us) server_negotiation.cc:407] Connection header received
0807 08:39:00.115484 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:00.115489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:00.115579 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:00.115712 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:00.117275 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.118006 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.119537 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.119800 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.119934 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:00.121466 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:00.121470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:00.121472 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:00.121488 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:00.121515 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:00.121524 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:00.121612 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:00.121793 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 08:39:00.121876 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 08:39:00.123570 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.112207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43681 (local address 127.9.174.2:46813)
0807 08:39:00.113110 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:00.113163 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:00.115180 (+ 2017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:00.116194 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:00.116203 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:00.116324 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:00.117105 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:00.117123 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.118174 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.118178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:00.119390 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:00.119402 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.119996 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.120001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:00.120208 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:00.121126 (+ 918us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:39:00.121127 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:00.121183 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:39:00.121268 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:00.121616 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:00.121629 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:00.121639 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 08:39:00.121794 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 08:39:00.122205 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":157,"threads_started":1}
I20260807 08:39:00.125092 5698 heartbeater.cc:347] Connected to a master server at 127.9.174.62:43681
I20260807 08:39:00.125427 5698 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:00.126220 5698 heartbeater.cc:511] Master 127.9.174.62:43681 requested a full tablet report, sending...
I20260807 08:39:00.127715 5241 ts_manager.cc:194] Registered new tserver with Master: 8cf5bdd38a5245b486800d3de10c52ec (127.9.174.2:44127)
I20260807 08:39:00.128504 5241 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.174.2:46813
W20260807 08:39:00.258273 5702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:00.258666 5702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:00.258749 5702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:00.264945 5702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:00.265055 5702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:00.265166 5702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:39:00.270591 5702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:44617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:43681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:00.272199 5702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:00.274168 5702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:39:00.286801 5702 server_base.cc:1034] running on GCE node
W20260807 08:39:00.286741 5710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:00.286749 5709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:00.286741 5713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:00.287528 5702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:00.288380 5702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:00.289606 5702 hybrid_clock.cc:648] HybridClock initialized: now 1786091940289587 us; error 32 us; skew 500 ppm
I20260807 08:39:00.292295 5702 webserver.cc:460] Webserver started at http://127.9.174.3:46611/ using document root <none> and password file <none>
I20260807 08:39:00.293138 5702 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:00.293248 5702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:00.293596 5702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:00.296371 5702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "e424f89f3840448eadf0fb802978bd7a"
format_stamp: "Formatted at 2026-08-07 08:39:00 on dist-test-slave-h3ft"
server_key: "fc5f327ffe5197f33856f2505f46d904"
server_key_iv: "ed728566ca741077fd9cfbb309bcd4c1"
server_key_version: "encryptionkey@0"
I20260807 08:39:00.297098 5702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "e424f89f3840448eadf0fb802978bd7a"
format_stamp: "Formatted at 2026-08-07 08:39:00 on dist-test-slave-h3ft"
server_key: "fc5f327ffe5197f33856f2505f46d904"
server_key_iv: "ed728566ca741077fd9cfbb309bcd4c1"
server_key_version: "encryptionkey@0"
I20260807 08:39:00.301858 5702 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 08:39:00.305013 5721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:00.306771 5702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:39:00.306981 5702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "e424f89f3840448eadf0fb802978bd7a"
format_stamp: "Formatted at 2026-08-07 08:39:00 on dist-test-slave-h3ft"
server_key: "fc5f327ffe5197f33856f2505f46d904"
server_key_iv: "ed728566ca741077fd9cfbb309bcd4c1"
server_key_version: "encryptionkey@0"
I20260807 08:39:00.307224 5702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:00.336256 5702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:00.337273 5702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:00.337548 5702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:00.338380 5702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:00.339716 5702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:00.339797 5702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:00.339905 5702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:00.339947 5702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:00.351701 5702 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:38799
I20260807 08:39:00.351814 5881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:38799 every 8 connection(s)
I20260807 08:39:00.353003 5702 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:39:00.359853 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 5702
I20260807 08:39:00.360045 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:39:00.360466 9912 external_mini_cluster.cc:1428] Setting key d6751855d47bbdd9127cd87a756cf32e
I20260807 08:39:00.365742 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.355488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:43297 (local address 127.9.174.62:43681)
0807 08:39:00.355671 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:00.355675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:00.357148 (+ 1473us) server_negotiation.cc:407] Connection header received
0807 08:39:00.358939 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:00.358942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:00.359093 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:00.359186 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:00.360708 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.361364 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.363192 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.363496 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.363588 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:00.365159 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:00.365163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:00.365165 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:00.365180 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:00.365203 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:00.365211 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:00.365292 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:00.365528 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 08:39:00.365598 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 08:39:00.366974 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.356004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:43681 (local address 127.9.174.3:43297)
0807 08:39:00.356947 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:00.357002 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:00.358677 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:00.359477 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:00.359488 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:00.359634 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:00.360475 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:00.360496 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.361562 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.361566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:00.363049 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:00.363068 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.363681 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.363685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:00.363875 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:00.364801 (+ 926us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:39:00.364803 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:00.364861 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:39:00.364966 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:00.365338 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:00.365349 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:00.365355 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:39:00.365496 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 08:39:00.365792 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":150,"threads_started":1}
I20260807 08:39:00.368432 5883 heartbeater.cc:347] Connected to a master server at 127.9.174.62:43681
I20260807 08:39:00.368767 5883 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:00.369509 5883 heartbeater.cc:511] Master 127.9.174.62:43681 requested a full tablet report, sending...
I20260807 08:39:00.371038 5241 ts_manager.cc:194] Registered new tserver with Master: e424f89f3840448eadf0fb802978bd7a (127.9.174.3:38799)
I20260807 08:39:00.371754 5241 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.174.3:43297
I20260807 08:39:00.375464 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:39:00.382804 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.378113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44510 (local address 127.9.174.62:43681)
0807 08:39:00.378320 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:00.378327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:00.378502 (+ 175us) server_negotiation.cc:407] Connection header received
0807 08:39:00.378678 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:00.378682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:00.378744 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:00.378840 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:00.379224 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.379830 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.380808 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.381055 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.381167 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:00.382185 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:00.382194 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:00.382196 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:00.382210 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:00.382235 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:00.382244 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:00.382413 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:00.382567 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 08:39:00.382613 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 08:39:00.386444 5241 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:39:00.390242 5241 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:39:00.407905 5907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.402376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:36139 (local address 127.0.0.1:54842)
0807 08:39:00.402861 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:00.402880 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:00.402987 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:00.403622 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:00.403625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:00.403644 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:00.403741 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:00.403747 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.405029 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.405032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:00.406192 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:00.406203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.407262 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.407266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:00.407384 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:00.407431 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 08:39:00.407556 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 08:39:00.407708 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":55,"thread_start_us":171,"threads_started":1}
I20260807 08:39:00.407899 5908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.402774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54842 (local address 127.9.174.1:36139)
0807 08:39:00.403249 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:00.403254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:00.403275 (+ 21us) server_negotiation.cc:407] Connection header received
0807 08:39:00.403345 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:00.403349 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:00.403474 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:00.403640 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:00.403883 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.404877 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.406326 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.407122 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.407244 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:00.407491 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:00.407588 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 08:39:00.407723 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":126,"threads_started":1}
I20260807 08:39:00.408378 5902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.400782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:44127 (local address 127.0.0.1:54790)
0807 08:39:00.401508 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:00.401570 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:00.401737 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:00.403080 (+ 1343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:00.403086 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:00.403123 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:00.403484 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:00.403492 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.405232 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.405235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:00.406568 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:00.406580 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.407904 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.407908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:00.408007 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:00.408035 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 08:39:00.408109 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 08:39:00.408218 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":105,"spinlock_wait_cycles":17024,"thread_start_us":97,"threads_started":1}
I20260807 08:39:00.408751 5900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.400842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:38799 (local address 127.0.0.1:44534)
0807 08:39:00.401508 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:00.401570 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:00.401737 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:00.403080 (+ 1343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:00.403086 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:00.403122 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:00.403484 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:00.403492 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.405387 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.405393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:00.406568 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:00.406580 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.408341 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.408345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:00.408432 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:00.408456 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 08:39:00.408536 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 08:39:00.408613 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":120,"spinlock_wait_cycles":10368,"thread_start_us":145,"threads_started":1}
I20260807 08:39:00.409061 5903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.400961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54790 (local address 127.9.174.2:44127)
0807 08:39:00.402367 (+ 1406us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:00.402379 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:00.402402 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:39:00.402521 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:00.402526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:00.402675 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:00.402852 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:00.403848 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.405093 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.406749 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.407563 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.407657 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:00.407842 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:00.408652 (+ 810us) server_negotiation.cc:299] Negotiation successful
0807 08:39:00.408775 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1283,"thread_start_us":157,"threads_started":1}
I20260807 08:39:00.409264 5906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.401663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44534 (local address 127.9.174.3:38799)
0807 08:39:00.402240 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:00.402254 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:00.402287 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:39:00.402403 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:00.402408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:00.402677 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:00.402852 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:00.403924 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.405265 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.406708 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.407590 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.407693 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:00.408908 (+ 1215us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:00.408964 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 08:39:00.409078 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":161,"threads_started":1}
I20260807 08:39:00.412264 5614 tablet_service.cc:1467] Processing CreateTablet for tablet c15df27e72b0495c9e2a9d841a523239 (DEFAULT_TABLE table=test-table [id=ac305769557144c7b8a57a23a66b7807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:39:00.412256 5793 tablet_service.cc:1467] Processing CreateTablet for tablet c15df27e72b0495c9e2a9d841a523239 (DEFAULT_TABLE table=test-table [id=ac305769557144c7b8a57a23a66b7807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:39:00.412274 5410 tablet_service.cc:1467] Processing CreateTablet for tablet c15df27e72b0495c9e2a9d841a523239 (DEFAULT_TABLE table=test-table [id=ac305769557144c7b8a57a23a66b7807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:39:00.413698 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c15df27e72b0495c9e2a9d841a523239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:00.413699 5793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c15df27e72b0495c9e2a9d841a523239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:00.414768 5410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c15df27e72b0495c9e2a9d841a523239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:00.421680 5909 tablet_bootstrap.cc:492] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec: Bootstrap starting.
I20260807 08:39:00.423158 5911 tablet_bootstrap.cc:492] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9: Bootstrap starting.
I20260807 08:39:00.423305 5910 tablet_bootstrap.cc:492] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a: Bootstrap starting.
I20260807 08:39:00.425345 5909 tablet_bootstrap.cc:654] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:00.425432 5911 tablet_bootstrap.cc:654] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:00.425431 5910 tablet_bootstrap.cc:654] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:00.426301 5909 log.cc:824] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:00.426342 5910 log.cc:824] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:00.426342 5911 log.cc:824] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:00.428625 5911 tablet_bootstrap.cc:492] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9: No bootstrap required, opened a new log
I20260807 08:39:00.428634 5910 tablet_bootstrap.cc:492] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a: No bootstrap required, opened a new log
I20260807 08:39:00.428642 5909 tablet_bootstrap.cc:492] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec: No bootstrap required, opened a new log
I20260807 08:39:00.428870 5909 ts_tablet_manager.cc:1397] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 08:39:00.428871 5910 ts_tablet_manager.cc:1397] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 08:39:00.428880 5911 ts_tablet_manager.cc:1397] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 08:39:00.433238 5911 raft_consensus.cc:348] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf5bdd38a5245b486800d3de10c52ec" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44127 } } peers { permanent_uuid: "e424f89f3840448eadf0fb802978bd7a" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38799 } } peers { permanent_uuid: "f5012659eacb4a51b1851ff2997867c9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36139 } }
I20260807 08:39:00.433604 5911 raft_consensus.cc:374] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:00.433697 5911 raft_consensus.cc:729] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5012659eacb4a51b1851ff2997867c9, State: Initialized, Role: FOLLOWER
I20260807 08:39:00.433588 5910 raft_consensus.cc:348] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf5bdd38a5245b486800d3de10c52ec" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44127 } } peers { permanent_uuid: "e424f89f3840448eadf0fb802978bd7a" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38799 } } peers { permanent_uuid: "f5012659eacb4a51b1851ff2997867c9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36139 } }
I20260807 08:39:00.433857 5910 raft_consensus.cc:374] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:00.433924 5910 raft_consensus.cc:729] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e424f89f3840448eadf0fb802978bd7a, State: Initialized, Role: FOLLOWER
I20260807 08:39:00.433909 5909 raft_consensus.cc:348] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf5bdd38a5245b486800d3de10c52ec" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44127 } } peers { permanent_uuid: "e424f89f3840448eadf0fb802978bd7a" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38799 } } peers { permanent_uuid: "f5012659eacb4a51b1851ff2997867c9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36139 } }
I20260807 08:39:00.434237 5909 raft_consensus.cc:374] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:00.434304 5909 raft_consensus.cc:729] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cf5bdd38a5245b486800d3de10c52ec, State: Initialized, Role: FOLLOWER
I20260807 08:39:00.434413 5911 consensus_queue.cc:260] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf5bdd38a5245b486800d3de10c52ec" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44127 } } peers { permanent_uuid: "e424f89f3840448eadf0fb802978bd7a" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38799 } } peers { permanent_uuid: "f5012659eacb4a51b1851ff2997867c9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36139 } }
I20260807 08:39:00.434414 5910 consensus_queue.cc:260] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf5bdd38a5245b486800d3de10c52ec" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44127 } } peers { permanent_uuid: "e424f89f3840448eadf0fb802978bd7a" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38799 } } peers { permanent_uuid: "f5012659eacb4a51b1851ff2997867c9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36139 } }
I20260807 08:39:00.434808 5909 consensus_queue.cc:260] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf5bdd38a5245b486800d3de10c52ec" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44127 } } peers { permanent_uuid: "e424f89f3840448eadf0fb802978bd7a" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38799 } } peers { permanent_uuid: "f5012659eacb4a51b1851ff2997867c9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36139 } }
I20260807 08:39:00.435348 5519 heartbeater.cc:503] Master 127.9.174.62:43681 was elected leader, sending a full tablet report...
I20260807 08:39:00.435637 5911 ts_tablet_manager.cc:1428] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9: Time spent starting tablet: real 0.007s user 0.002s sys 0.006s
I20260807 08:39:00.435639 5910 ts_tablet_manager.cc:1428] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 08:39:00.435734 5916 raft_consensus.cc:482] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:39:00.435756 5909 ts_tablet_manager.cc:1428] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 08:39:00.436005 5883 heartbeater.cc:503] Master 127.9.174.62:43681 was elected leader, sending a full tablet report...
I20260807 08:39:00.436072 5698 heartbeater.cc:503] Master 127.9.174.62:43681 was elected leader, sending a full tablet report...
I20260807 08:39:00.439060 5916 raft_consensus.cc:504] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf5bdd38a5245b486800d3de10c52ec" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44127 } } peers { permanent_uuid: "e424f89f3840448eadf0fb802978bd7a" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38799 } } peers { permanent_uuid: "f5012659eacb4a51b1851ff2997867c9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36139 } }
I20260807 08:39:00.441180 5916 leader_election.cc:288] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cf5bdd38a5245b486800d3de10c52ec (127.9.174.2:44127), e424f89f3840448eadf0fb802978bd7a (127.9.174.3:38799)
I20260807 08:39:00.447062 5906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.442066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:60143 (local address 127.9.174.3:38799)
0807 08:39:00.442196 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:00.442200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:00.442514 (+ 314us) server_negotiation.cc:407] Connection header received
0807 08:39:00.442639 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:00.442645 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:00.442689 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:00.442758 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:00.443253 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.444053 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.446007 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.446427 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.446519 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:00.446582 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:00.446797 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 08:39:00.446904 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:39:00.447098 5921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.441657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:38799 (local address 127.9.174.1:60143)
0807 08:39:00.442416 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:00.442428 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:00.442550 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:00.442900 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:00.442903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:00.442920 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:00.443101 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:00.443114 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.444222 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.444225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:00.445850 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:00.445863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.446545 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.446547 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:00.446641 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:00.446666 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 08:39:00.446775 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 08:39:00.446860 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":77,"thread_start_us":136,"threads_started":1}
I20260807 08:39:00.447880 5922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.441530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:44127 (local address 127.9.174.1:46747)
0807 08:39:00.441951 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:00.441969 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:00.442153 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:00.442602 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:00.442605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:00.442624 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:00.442798 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:00.442806 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.444043 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.444046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:00.445850 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:00.445863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.447397 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.447400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:00.447523 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:00.447564 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 08:39:00.447641 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 08:39:00.447710 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":73,"thread_start_us":145,"threads_started":1}
I20260807 08:39:00.447896 5903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.441587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:46747 (local address 127.9.174.2:44127)
0807 08:39:00.441731 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:00.441736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:00.442066 (+ 330us) server_negotiation.cc:407] Connection header received
0807 08:39:00.442292 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:00.442297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:00.442366 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:00.442417 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:00.442928 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.443908 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.446636 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.447149 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.447273 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:00.447334 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:00.447689 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 08:39:00.447774 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:39:00.448103 5822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c15df27e72b0495c9e2a9d841a523239" candidate_uuid: "f5012659eacb4a51b1851ff2997867c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e424f89f3840448eadf0fb802978bd7a" is_pre_election: true
I20260807 08:39:00.448505 5822 raft_consensus.cc:2454] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5012659eacb4a51b1851ff2997867c9 in term 0.
I20260807 08:39:00.448601 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c15df27e72b0495c9e2a9d841a523239" candidate_uuid: "f5012659eacb4a51b1851ff2997867c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cf5bdd38a5245b486800d3de10c52ec" is_pre_election: true
I20260807 08:39:00.448985 5640 raft_consensus.cc:2454] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5012659eacb4a51b1851ff2997867c9 in term 0.
I20260807 08:39:00.449064 5345 leader_election.cc:302] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e424f89f3840448eadf0fb802978bd7a, f5012659eacb4a51b1851ff2997867c9; no voters:
I20260807 08:39:00.449496 5916 raft_consensus.cc:2781] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:39:00.449610 5916 raft_consensus.cc:482] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:39:00.449662 5916 raft_consensus.cc:3037] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:00.451272 5916 raft_consensus.cc:504] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf5bdd38a5245b486800d3de10c52ec" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44127 } } peers { permanent_uuid: "e424f89f3840448eadf0fb802978bd7a" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38799 } } peers { permanent_uuid: "f5012659eacb4a51b1851ff2997867c9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36139 } }
I20260807 08:39:00.451805 5916 leader_election.cc:288] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [CANDIDATE]: Term 1 election: Requested vote from peers 8cf5bdd38a5245b486800d3de10c52ec (127.9.174.2:44127), e424f89f3840448eadf0fb802978bd7a (127.9.174.3:38799)
I20260807 08:39:00.452196 5640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c15df27e72b0495c9e2a9d841a523239" candidate_uuid: "f5012659eacb4a51b1851ff2997867c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cf5bdd38a5245b486800d3de10c52ec"
I20260807 08:39:00.452346 5640 raft_consensus.cc:3037] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:00.453768 5640 raft_consensus.cc:2454] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5012659eacb4a51b1851ff2997867c9 in term 1.
I20260807 08:39:00.454085 5822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c15df27e72b0495c9e2a9d841a523239" candidate_uuid: "f5012659eacb4a51b1851ff2997867c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e424f89f3840448eadf0fb802978bd7a"
I20260807 08:39:00.454183 5346 leader_election.cc:302] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cf5bdd38a5245b486800d3de10c52ec, f5012659eacb4a51b1851ff2997867c9; no voters:
I20260807 08:39:00.454387 5916 raft_consensus.cc:2781] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:00.454339 5822 raft_consensus.cc:3037] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:00.454516 5916 raft_consensus.cc:686] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [term 1 LEADER]: Becoming Leader. State: Replica: f5012659eacb4a51b1851ff2997867c9, State: Running, Role: LEADER
I20260807 08:39:00.454921 5916 consensus_queue.cc:237] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf5bdd38a5245b486800d3de10c52ec" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44127 } } peers { permanent_uuid: "e424f89f3840448eadf0fb802978bd7a" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38799 } } peers { permanent_uuid: "f5012659eacb4a51b1851ff2997867c9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36139 } }
I20260807 08:39:00.455914 5822 raft_consensus.cc:2454] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5012659eacb4a51b1851ff2997867c9 in term 1.
I20260807 08:39:00.458962 5239 catalog_manager.cc:5495] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5012659eacb4a51b1851ff2997867c9 (127.9.174.1). New cstate: current_term: 1 leader_uuid: "f5012659eacb4a51b1851ff2997867c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf5bdd38a5245b486800d3de10c52ec" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e424f89f3840448eadf0fb802978bd7a" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5012659eacb4a51b1851ff2997867c9" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 36139 } health_report { overall_health: HEALTHY } } }
I20260807 08:39:00.475713 5908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.471393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54844 (local address 127.9.174.1:36139)
0807 08:39:00.471566 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:00.471571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:00.471591 (+ 20us) server_negotiation.cc:407] Connection header received
0807 08:39:00.471704 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:00.471707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:00.471775 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:00.471904 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:39:00.472206 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.472796 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.473816 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:00.474053 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:00.474168 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:00.474598 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:39:00.475242 (+ 644us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:39:00.475425 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:00.475490 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 08:39:00.475534 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 08:39:00.484280 5640 raft_consensus.cc:1261] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec [term 1 FOLLOWER]: Refusing update from remote peer f5012659eacb4a51b1851ff2997867c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:39:00.484280 5822 raft_consensus.cc:1261] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a [term 1 FOLLOWER]: Refusing update from remote peer f5012659eacb4a51b1851ff2997867c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:39:00.485147 5916 consensus_queue.cc:1035] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e424f89f3840448eadf0fb802978bd7a" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:39:00.485399 5917 consensus_queue.cc:1035] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cf5bdd38a5245b486800d3de10c52ec" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 44127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:39:00.498172 5927 mvcc.cc:205] Tried to move back new op lower bound from 7315832588214005760 to 7315832588107743232. Current Snapshot: MvccSnapshot[applied={T|T < 7315832588214005760 or (T in {7315832588214005760})}]
I20260807 08:39:00.508395 5239 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:39:00.508697 5239 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:39:00.512310 5239 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6571789447dd4a4e93c91b26373aef37: Sending DeleteTablet for 3 replicas of tablet c15df27e72b0495c9e2a9d841a523239
I20260807 08:39:00.513587 5793 tablet_service.cc:1514] Processing DeleteTablet for tablet c15df27e72b0495c9e2a9d841a523239 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:39:00 UTC) from {username='slave'} at 127.0.0.1:44534
I20260807 08:39:00.513590 5614 tablet_service.cc:1514] Processing DeleteTablet for tablet c15df27e72b0495c9e2a9d841a523239 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:39:00 UTC) from {username='slave'} at 127.0.0.1:54790
I20260807 08:39:00.513671 5410 tablet_service.cc:1514] Processing DeleteTablet for tablet c15df27e72b0495c9e2a9d841a523239 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:39:00 UTC) from {username='slave'} at 127.0.0.1:54842
I20260807 08:39:00.514397 5944 tablet_replica.cc:330] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec: stopping tablet replica
I20260807 08:39:00.514703 5943 tablet_replica.cc:330] T c15df27e72b0495c9e2a9d841a523239 P f5012659eacb4a51b1851ff2997867c9: stopping tablet replica
I20260807 08:39:00.514896 5944 raft_consensus.cc:2229] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:39:00.515489 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 5320
I20260807 08:39:00.515380 5944 raft_consensus.cc:2258] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:39:00.514921 5940 tablet_replica.cc:330] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a: stopping tablet replica
I20260807 08:39:00.527456 5940 raft_consensus.cc:2229] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:39:00.527953 5940 raft_consensus.cc:2258] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:39:00.528412 5944 ts_tablet_manager.cc:1905] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:39:00.532800 5940 ts_tablet_manager.cc:1905] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:39:00.532841 5944 ts_tablet_manager.cc:1918] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:39:00.532958 5944 log.cc:1196] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/c15df27e72b0495c9e2a9d841a523239
I20260807 08:39:00.533349 5944 ts_tablet_manager.cc:1939] T c15df27e72b0495c9e2a9d841a523239 P 8cf5bdd38a5245b486800d3de10c52ec: Deleting consensus metadata
I20260807 08:39:00.535130 5222 catalog_manager.cc:4841] TS 8cf5bdd38a5245b486800d3de10c52ec (127.9.174.2:44127): tablet c15df27e72b0495c9e2a9d841a523239 (table test-table [id=ac305769557144c7b8a57a23a66b7807]) successfully deleted
I20260807 08:39:00.536284 5940 ts_tablet_manager.cc:1918] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:39:00.536408 5940 log.cc:1196] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/c15df27e72b0495c9e2a9d841a523239
I20260807 08:39:00.536739 5940 ts_tablet_manager.cc:1939] T c15df27e72b0495c9e2a9d841a523239 P e424f89f3840448eadf0fb802978bd7a: Deleting consensus metadata
I20260807 08:39:00.538079 5220 catalog_manager.cc:4841] TS e424f89f3840448eadf0fb802978bd7a (127.9.174.3:38799): tablet c15df27e72b0495c9e2a9d841a523239 (table test-table [id=ac305769557144c7b8a57a23a66b7807]) successfully deleted
W20260807 08:39:00.538427 5220 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:36139 (error 108)
I20260807 08:39:00.538861 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 5523
I20260807 08:39:00.543576 5900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:00.543175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:36139 (local address 127.0.0.1:54848)
0807 08:39:00.543366 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:00.543461 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 08:39:00.543794 5220 catalog_manager.cc:4584] TS f5012659eacb4a51b1851ff2997867c9 (127.9.174.1:36139): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c15df27e72b0495c9e2a9d841a523239: Network error: Client connection negotiation failed: client connection to 127.9.174.1:36139: connect: Connection refused (error 111)
I20260807 08:39:00.550872 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 5702
I20260807 08:39:00.566861 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 5194
2026-08-07T08:39:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1308 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:00 dist-test-slave-h3ft krb5kdc[5950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:00 dist-test-slave-h3ft krb5kdc[5950](info): set up 2 sockets
Aug 07 08:39:00 dist-test-slave-h3ft krb5kdc[5950](info): commencing operation
krb5kdc: starting...
W20260807 08:39:01.869161 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:01 dist-test-slave-h3ft krb5kdc[5950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:39:02.019133 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:40959
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:35891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:40959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:02.157073 5968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:02.157533 5968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:02.157621 5968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:02.162106 5968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:02.162235 5968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:02.162272 5968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:02.162313 5968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:02.162349 5968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:02.168696 5968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:35891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:40959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:40959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:02.170282 5968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:02.171689 5968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:02.183063 5973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:02.183169 5968 server_base.cc:1034] running on GCE node
W20260807 08:39:02.183053 5974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:02.183053 5979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:02.184271 5968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:02.185799 5968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:02.187042 5968 hybrid_clock.cc:648] HybridClock initialized: now 1786091942186993 us; error 33 us; skew 500 ppm
Aug 07 08:39:02 dist-test-slave-h3ft krb5kdc[5950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091942, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:02.190601 5968 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:39:02.191948 5968 webserver.cc:460] Webserver started at http://127.9.174.62:34643/ using document root <none> and password file <none>
I20260807 08:39:02.192732 5968 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:02.192837 5968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:02.193138 5968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:02.195974 5968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "7734b06cbc5d43eab2dade27980ccdc7"
format_stamp: "Formatted at 2026-08-07 08:39:02 on dist-test-slave-h3ft"
server_key: "8316d973f6dca035c8b2b6b47735066f"
server_key_iv: "716a71e2e0be6f7dd4eede6fff7176b7"
server_key_version: "encryptionkey@0"
I20260807 08:39:02.196734 5968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "7734b06cbc5d43eab2dade27980ccdc7"
format_stamp: "Formatted at 2026-08-07 08:39:02 on dist-test-slave-h3ft"
server_key: "8316d973f6dca035c8b2b6b47735066f"
server_key_iv: "716a71e2e0be6f7dd4eede6fff7176b7"
server_key_version: "encryptionkey@0"
I20260807 08:39:02.201920 5968 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:39:02.205286 5988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:02.207202 5968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:39:02.207429 5968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "7734b06cbc5d43eab2dade27980ccdc7"
format_stamp: "Formatted at 2026-08-07 08:39:02 on dist-test-slave-h3ft"
server_key: "8316d973f6dca035c8b2b6b47735066f"
server_key_iv: "716a71e2e0be6f7dd4eede6fff7176b7"
server_key_version: "encryptionkey@0"
I20260807 08:39:02.207610 5968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:02.227283 5968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:02.231746 5968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:02.232054 5968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:02.241330 5968 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:40959
I20260807 08:39:02.241353 6070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:40959 every 8 connection(s)
I20260807 08:39:02.242607 5968 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:02.246399 6072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:02.249770 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 5968
I20260807 08:39:02.249949 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:02.250349 9912 external_mini_cluster.cc:1428] Setting key a93cf359dcf68a1fe2989c9e5d1f2c45
I20260807 08:39:02.254969 6072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7: Bootstrap starting.
I20260807 08:39:02.258072 6072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:02.259133 6072 log.cc:824] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:39:02 dist-test-slave-h3ft krb5kdc[5950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:39:02.262344 6072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7: No bootstrap required, opened a new log
I20260807 08:39:02.268230 6072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7734b06cbc5d43eab2dade27980ccdc7" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40959 } }
I20260807 08:39:02.268575 6072 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:02.268646 6072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7734b06cbc5d43eab2dade27980ccdc7, State: Initialized, Role: FOLLOWER
I20260807 08:39:02.269121 6078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:02.252345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56056 (local address 127.9.174.62:40959)
0807 08:39:02.252970 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:02.252994 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:02.253046 (+ 52us) server_negotiation.cc:407] Connection header received
0807 08:39:02.253869 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:02.253890 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:02.254328 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:02.254773 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:02.255793 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.256902 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.257838 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.258531 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.258717 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:02.262421 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:02.262432 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:02.262459 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:02.262512 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:02.264997 (+ 2485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:02.265746 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:02.265754 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:02.265762 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:02.265955 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:02.266326 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:02.266329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:02.266330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:02.266905 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:02.267222 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:02.267552 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 08:39:02.267979 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":211,"threads_started":1}
I20260807 08:39:02.269187 6072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7734b06cbc5d43eab2dade27980ccdc7" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40959 } }
I20260807 08:39:02.269390 6072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:02.269457 6072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:02.269551 6072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:02.270817 6072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7734b06cbc5d43eab2dade27980ccdc7" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40959 } }
I20260807 08:39:02.271281 6072 leader_election.cc:302] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7734b06cbc5d43eab2dade27980ccdc7; no voters:
I20260807 08:39:02.271698 6072 leader_election.cc:288] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:02.271808 6080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:02.272025 6080 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [term 1 LEADER]: Becoming Leader. State: Replica: 7734b06cbc5d43eab2dade27980ccdc7, State: Running, Role: LEADER
I20260807 08:39:02.272398 6080 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7734b06cbc5d43eab2dade27980ccdc7" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40959 } }
I20260807 08:39:02.273490 6072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:02.274622 6083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7734b06cbc5d43eab2dade27980ccdc7. Latest consensus state: current_term: 1 leader_uuid: "7734b06cbc5d43eab2dade27980ccdc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7734b06cbc5d43eab2dade27980ccdc7" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40959 } } }
I20260807 08:39:02.274641 6082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7734b06cbc5d43eab2dade27980ccdc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7734b06cbc5d43eab2dade27980ccdc7" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 40959 } } }
I20260807 08:39:02.274932 6082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:02.275350 6083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:02.275516 6094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:02.279724 6094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:02.287775 6094 catalog_manager.cc:1347] Generated new cluster ID: ff488b5fb74f46ec897bcd77daf42d92
I20260807 08:39:02.287899 6094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:02.299592 6094 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:02.301213 6094 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:02.312417 6094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7: Generated new TSK 0
I20260807 08:39:02.313607 6094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.174.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
Entry for principal kudu/127.9.174.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab.
I20260807 08:39:02.356215 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:40959
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:35891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:02.505793 6111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:02.506237 6111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:02.506474 6111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:02.511898 6111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:02.512010 6111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:02.512117 6111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:39:02.518781 6111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:35891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.1.keytab
--principal=kudu/127.9.174.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:40959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:02.520515 6111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:02.522326 6111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:02.536163 6121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:02.536164 6118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:02.536538 6111 server_base.cc:1034] running on GCE node
W20260807 08:39:02.536163 6119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:02.537305 6111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:02.538205 6111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:02.539410 6111 hybrid_clock.cc:648] HybridClock initialized: now 1786091942539382 us; error 44 us; skew 500 ppm
Aug 07 08:39:02 dist-test-slave-h3ft krb5kdc[5950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091942, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:02.543237 6111 init.cc:377] Logged in from keytab as kudu/127.9.174.1@KRBTEST.COM (short username kudu)
I20260807 08:39:02.544473 6111 webserver.cc:460] Webserver started at http://127.9.174.1:34229/ using document root <none> and password file <none>
I20260807 08:39:02.545234 6111 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:02.545336 6111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:02.545631 6111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:02.548403 6111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "0f11bf2498e34ce4ab9382ea2dfb4f09"
format_stamp: "Formatted at 2026-08-07 08:39:02 on dist-test-slave-h3ft"
server_key: "7ef04f362ceb732ada32cc3e5652b68d"
server_key_iv: "7e4784a4191304e4f01b36cae4be3cee"
server_key_version: "encryptionkey@0"
I20260807 08:39:02.549134 6111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "0f11bf2498e34ce4ab9382ea2dfb4f09"
format_stamp: "Formatted at 2026-08-07 08:39:02 on dist-test-slave-h3ft"
server_key: "7ef04f362ceb732ada32cc3e5652b68d"
server_key_iv: "7e4784a4191304e4f01b36cae4be3cee"
server_key_version: "encryptionkey@0"
I20260807 08:39:02.554316 6111 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:39:02.557952 6128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:02.559792 6111 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:39:02.559988 6111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "0f11bf2498e34ce4ab9382ea2dfb4f09"
format_stamp: "Formatted at 2026-08-07 08:39:02 on dist-test-slave-h3ft"
server_key: "7ef04f362ceb732ada32cc3e5652b68d"
server_key_iv: "7e4784a4191304e4f01b36cae4be3cee"
server_key_version: "encryptionkey@0"
I20260807 08:39:02.560168 6111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:02.581815 6111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:02.586627 6111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:02.586908 6111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:02.587800 6111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:02.588976 6111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:02.589056 6111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:02.589155 6111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:02.589196 6111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:02.601267 6111 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:37131
I20260807 08:39:02.601289 6288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:37131 every 8 connection(s)
I20260807 08:39:02.602583 6111 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:39:02.608091 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 6111
I20260807 08:39:02.608281 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:39:02.608765 9912 external_mini_cluster.cc:1428] Setting key 54da651c06c15900f018e6147c789ca7
Aug 07 08:39:02 dist-test-slave-h3ft krb5kdc[5950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091942, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.1@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:39:02.622970 6078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:02.605089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:58799 (local address 127.9.174.62:40959)
0807 08:39:02.605269 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:02.605273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:02.606465 (+ 1192us) server_negotiation.cc:407] Connection header received
0807 08:39:02.608481 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:02.608486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:02.608571 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:02.608687 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:02.610715 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.611537 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.613789 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.614248 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.614398 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:02.618594 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:02.618601 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:02.618606 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:02.618652 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:02.620392 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:02.621054 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:02.621062 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:02.621064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:02.621156 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:02.621555 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:02.621559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:02.621561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:02.621799 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:02.621913 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:02.622577 (+ 664us) server_negotiation.cc:299] Negotiation successful
0807 08:39:02.622756 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
WARNING: no policy specified for kudu/127.9.174.2@KRBTEST.COM; defaulting to no policy
I20260807 08:39:02.624229 6293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:02.605494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:40959 (local address 127.9.174.1:58799)
0807 08:39:02.606224 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:02.606287 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:02.608041 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:02.609014 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:02.609027 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:02.609698 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:02.610514 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:02.610545 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.611714 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.611773 (+ 59us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:02.613566 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:02.613587 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.614474 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.614478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:02.614697 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:02.615706 (+ 1009us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:02.615734 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:02.618288 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:02.620550 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:02.620558 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:02.620582 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:02.620925 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:02.621255 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:02.621259 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:02.621261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:02.621448 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:02.621914 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:02.621925 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:02.622291 (+ 366us) client_negotiation.cc:769] Sending connection context
0807 08:39:02.622536 (+ 245us) client_negotiation.cc:240] Negotiation successful
0807 08:39:02.623028 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":172,"threads_started":1}
I20260807 08:39:02.625900 6291 heartbeater.cc:347] Connected to a master server at 127.9.174.62:40959
I20260807 08:39:02.626287 6291 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:02.627133 6291 heartbeater.cc:511] Master 127.9.174.62:40959 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.2@KRBTEST.COM" created.
I20260807 08:39:02.629325 6012 ts_manager.cc:194] Registered new tserver with Master: 0f11bf2498e34ce4ab9382ea2dfb4f09 (127.9.174.1:37131)
I20260807 08:39:02.631330 6012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.1@KRBTEST.COM'} at 127.9.174.1:58799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
Entry for principal kudu/127.9.174.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab.
I20260807 08:39:02.647305 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:40959
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:35891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:02.792811 6297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:02.793170 6297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:02.793219 6297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:02.797588 6297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:02.797676 6297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:02.797747 6297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:39:02.803990 6297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:35891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.2.keytab
--principal=kudu/127.9.174.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:40959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:02.805415 6297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:02.806707 6297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:02.819397 6304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:02.819399 6303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:02.819581 6297 server_base.cc:1034] running on GCE node
W20260807 08:39:02.819399 6307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:02.820282 6297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:02.821156 6297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:02.822379 6297 hybrid_clock.cc:648] HybridClock initialized: now 1786091942822356 us; error 33 us; skew 500 ppm
Aug 07 08:39:02 dist-test-slave-h3ft krb5kdc[5950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091942, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:02.826035 6297 init.cc:377] Logged in from keytab as kudu/127.9.174.2@KRBTEST.COM (short username kudu)
I20260807 08:39:02.827345 6297 webserver.cc:460] Webserver started at http://127.9.174.2:35301/ using document root <none> and password file <none>
I20260807 08:39:02.828116 6297 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:02.828214 6297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:02.828490 6297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:02.831290 6297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "37e5227d6f5045cd8000a368ad712b5a"
format_stamp: "Formatted at 2026-08-07 08:39:02 on dist-test-slave-h3ft"
server_key: "cc1cccc371c711875eaba8576b29e282"
server_key_iv: "fd9392810175d45ffcd9640b3714272e"
server_key_version: "encryptionkey@0"
I20260807 08:39:02.832010 6297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "37e5227d6f5045cd8000a368ad712b5a"
format_stamp: "Formatted at 2026-08-07 08:39:02 on dist-test-slave-h3ft"
server_key: "cc1cccc371c711875eaba8576b29e282"
server_key_iv: "fd9392810175d45ffcd9640b3714272e"
server_key_version: "encryptionkey@0"
I20260807 08:39:02.837414 6297 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:39:02.840632 6316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:02.842330 6297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 08:39:02.842506 6297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "37e5227d6f5045cd8000a368ad712b5a"
format_stamp: "Formatted at 2026-08-07 08:39:02 on dist-test-slave-h3ft"
server_key: "cc1cccc371c711875eaba8576b29e282"
server_key_iv: "fd9392810175d45ffcd9640b3714272e"
server_key_version: "encryptionkey@0"
I20260807 08:39:02.842665 6297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:02.872365 6297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:02.877120 6297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:02.877398 6297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:02.878245 6297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:02.879365 6297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:02.879418 6297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:02.879493 6297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:02.879534 6297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:02.890960 6297 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:39813
I20260807 08:39:02.891069 6476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:39813 every 8 connection(s)
I20260807 08:39:02.892212 6297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:39:02.898684 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 6297
I20260807 08:39:02.898866 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:39:02.899314 9912 external_mini_cluster.cc:1428] Setting key e636e6e95bed3bad7481827d4103c8a8
Aug 07 08:39:02 dist-test-slave-h3ft krb5kdc[5950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091942, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.2@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:39:02.910528 6078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:02.894682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:41647 (local address 127.9.174.62:40959)
0807 08:39:02.894863 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:02.894868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:02.896028 (+ 1160us) server_negotiation.cc:407] Connection header received
0807 08:39:02.897645 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:02.897650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:02.897729 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:02.897838 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:02.899522 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.900258 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.901664 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.901921 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.902038 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:02.905447 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:02.905451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:02.905454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:02.905484 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:02.907452 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:02.908161 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:02.908168 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:02.908173 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:02.908317 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:02.908828 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:02.908834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:02.908838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:02.909162 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:02.909285 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:02.910013 (+ 728us) server_negotiation.cc:299] Negotiation successful
0807 08:39:02.910211 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 08:39:02.911648 6484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:02.895206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:40959 (local address 127.9.174.2:41647)
0807 08:39:02.895798 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:02.895851 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:02.897384 (+ 1533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:02.898068 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:02.898076 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:02.898581 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:02.899325 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:02.899344 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.900429 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.900433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:02.901514 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:02.901525 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.902117 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.902122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:02.902337 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:02.903163 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:02.903182 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:02.905200 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:02.907622 (+ 2422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:02.907628 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:02.907650 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:02.908014 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:02.908434 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:02.908436 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:02.908437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:02.908695 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:02.909289 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:02.909296 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:02.909688 (+ 392us) client_negotiation.cc:769] Sending connection context
0807 08:39:02.909889 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 08:39:02.910277 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":146,"threads_started":1}
I20260807 08:39:02.913298 6479 heartbeater.cc:347] Connected to a master server at 127.9.174.62:40959
I20260807 08:39:02.913625 6479 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.174.3@KRBTEST.COM; defaulting to no policy
I20260807 08:39:02.914326 6479 heartbeater.cc:511] Master 127.9.174.62:40959 requested a full tablet report, sending...
I20260807 08:39:02.915971 6012 ts_manager.cc:194] Registered new tserver with Master: 37e5227d6f5045cd8000a368ad712b5a (127.9.174.2:39813)
I20260807 08:39:02.916774 6012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.2@KRBTEST.COM'} at 127.9.174.2:41647
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
Entry for principal kudu/127.9.174.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab.
I20260807 08:39:02.939311 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:40959
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:35891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:03.081341 6488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:03.081808 6488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:03.081897 6488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:03.086508 6488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:03.086643 6488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:03.086756 6488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:39:03.093171 6488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:35891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.3.keytab
--principal=kudu/127.9.174.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:40959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:03.094815 6488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:03.096318 6488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:03.109548 6501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:03.109527 6497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:03.109527 6499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:03.109887 6488 server_base.cc:1034] running on GCE node
I20260807 08:39:03.110569 6488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:03.111702 6488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:03.112927 6488 hybrid_clock.cc:648] HybridClock initialized: now 1786091943112902 us; error 35 us; skew 500 ppm
Aug 07 08:39:03 dist-test-slave-h3ft krb5kdc[5950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091943, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:03.116819 6488 init.cc:377] Logged in from keytab as kudu/127.9.174.3@KRBTEST.COM (short username kudu)
I20260807 08:39:03.118175 6488 webserver.cc:460] Webserver started at http://127.9.174.3:34169/ using document root <none> and password file <none>
I20260807 08:39:03.119068 6488 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:03.119201 6488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:03.119489 6488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:03.122229 6488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "1970aa9a90b24fb9a366455c0a4e00f8"
format_stamp: "Formatted at 2026-08-07 08:39:03 on dist-test-slave-h3ft"
server_key: "22ef73555b28b87adffa9edc26fa9554"
server_key_iv: "8eaeae84294e870a7bdb71ce0f0c3ddc"
server_key_version: "encryptionkey@0"
I20260807 08:39:03.122973 6488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "1970aa9a90b24fb9a366455c0a4e00f8"
format_stamp: "Formatted at 2026-08-07 08:39:03 on dist-test-slave-h3ft"
server_key: "22ef73555b28b87adffa9edc26fa9554"
server_key_iv: "8eaeae84294e870a7bdb71ce0f0c3ddc"
server_key_version: "encryptionkey@0"
I20260807 08:39:03.128293 6488 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 08:39:03.131816 6513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:03.133671 6488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:39:03.133854 6488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "1970aa9a90b24fb9a366455c0a4e00f8"
format_stamp: "Formatted at 2026-08-07 08:39:03 on dist-test-slave-h3ft"
server_key: "22ef73555b28b87adffa9edc26fa9554"
server_key_iv: "8eaeae84294e870a7bdb71ce0f0c3ddc"
server_key_version: "encryptionkey@0"
I20260807 08:39:03.134045 6488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:03.152050 6488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:03.156505 6488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:03.156807 6488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:03.157635 6488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:03.158748 6488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:03.158824 6488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:03.158919 6488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:03.158962 6488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:03.170919 6488 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:34149
I20260807 08:39:03.170998 6657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:34149 every 8 connection(s)
I20260807 08:39:03.172151 6488 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:39:03.179919 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 6488
I20260807 08:39:03.180136 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:39:03.180528 9912 external_mini_cluster.cc:1428] Setting key 08c5597f71029250f5d0b4f60cd0bf7e
Aug 07 08:39:03 dist-test-slave-h3ft krb5kdc[5950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091943, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.3@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:39:03.190963 6078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.174764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:44429 (local address 127.9.174.62:40959)
0807 08:39:03.175019 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.175038 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.176240 (+ 1202us) server_negotiation.cc:407] Connection header received
0807 08:39:03.177897 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.177901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.177983 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.178088 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:03.179742 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.180557 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.182287 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.182647 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.182783 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.186368 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:03.186372 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:03.186374 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:03.186411 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:03.188140 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:03.188809 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:03.188812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:03.188813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:03.188916 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:03.189401 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:03.189404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:03.189406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:03.189675 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:03.189777 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.190543 (+ 766us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.190724 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 08:39:03.192144 6660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.175343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:40959 (local address 127.9.174.3:44429)
0807 08:39:03.176058 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:03.176113 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:03.177625 (+ 1512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:03.178310 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:03.178319 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:03.178801 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:03.179557 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.179576 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.180751 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.180754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.182103 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.182116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.182865 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.182868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.183247 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.184197 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:03.184221 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:03.186124 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:03.188283 (+ 2159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:03.188290 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:03.188309 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:03.188677 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:03.189010 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:03.189013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:03.189014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:03.189249 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:03.189800 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:03.189810 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:03.190215 (+ 405us) client_negotiation.cc:769] Sending connection context
0807 08:39:03.190452 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 08:39:03.190848 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":152,"threads_started":1}
I20260807 08:39:03.193905 6658 heartbeater.cc:347] Connected to a master server at 127.9.174.62:40959
I20260807 08:39:03.194288 6658 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:03.195187 6658 heartbeater.cc:511] Master 127.9.174.62:40959 requested a full tablet report, sending...
I20260807 08:39:03.196589 6012 ts_manager.cc:194] Registered new tserver with Master: 1970aa9a90b24fb9a366455c0a4e00f8 (127.9.174.3:34149)
I20260807 08:39:03.197314 6012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.174.3@KRBTEST.COM'} at 127.9.174.3:44429
I20260807 08:39:03.208446 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:39:03.220539 6078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.211092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56066 (local address 127.9.174.62:40959)
0807 08:39:03.211260 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.211265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.211470 (+ 205us) server_negotiation.cc:407] Connection header received
0807 08:39:03.211616 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.211619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.211733 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.211822 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:03.212748 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.213564 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.214571 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.214817 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.214922 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.216681 (+ 1759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:03.216684 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:03.216693 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:03.216723 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:03.218488 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:03.218988 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:03.218991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:03.218992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:03.219158 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:03.219495 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:03.219498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:03.219499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:03.219790 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:03.219964 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.220202 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.220349 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 08:39:03.224588 6012 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:39:03.228828 6012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:39:03.248651 6669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.240602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:39813 (local address 127.0.0.1:40350)
0807 08:39:03.241151 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:03.241214 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:03.241398 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:03.242412 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:03.242423 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:03.242461 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:03.242665 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.242675 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.244434 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.244438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.246080 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.246091 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.247759 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.247763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.247908 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.247977 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 08:39:03.248098 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 08:39:03.248331 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":1,"thread_start_us":128,"threads_started":1}
I20260807 08:39:03.248927 6677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.242742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:37131 (local address 127.0.0.1:41136)
0807 08:39:03.243368 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:03.243385 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:03.243496 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:03.244234 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:03.244237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:03.244267 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:03.244403 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.244409 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.245847 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.245851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.247167 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.247180 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.248483 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.248488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.248594 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.248619 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 08:39:03.248690 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 08:39:03.248761 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":48,"spinlock_wait_cycles":896,"thread_start_us":103,"threads_started":1}
I20260807 08:39:03.249285 6671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.240814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40350 (local address 127.9.174.2:39813)
0807 08:39:03.241484 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.241502 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.241534 (+ 32us) server_negotiation.cc:407] Connection header received
0807 08:39:03.241636 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.241643 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.241923 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.242116 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:03.242837 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.244263 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.246240 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.247395 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.247508 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.247692 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.248918 (+ 1226us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.249064 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":202,"threads_started":1}
I20260807 08:39:03.249464 6675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.242304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41136 (local address 127.9.174.1:37131)
0807 08:39:03.243227 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.243234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.243600 (+ 366us) server_negotiation.cc:407] Connection header received
0807 08:39:03.243698 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.243703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.243917 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.244060 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:03.244535 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.245709 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.247311 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.248159 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.248267 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.248416 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.249111 (+ 695us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.249228 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":166,"threads_started":1}
I20260807 08:39:03.250648 6674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.242076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40972 (local address 127.9.174.3:34149)
0807 08:39:03.242982 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.243022 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.243316 (+ 294us) server_negotiation.cc:407] Connection header received
0807 08:39:03.243594 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.243601 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.243831 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.243972 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:03.244517 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.245869 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.248021 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.248945 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.249034 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.249181 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.250318 (+ 1137us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.250449 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":420,"threads_started":1}
I20260807 08:39:03.251138 6673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.241585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:34149 (local address 127.0.0.1:40972)
0807 08:39:03.243198 (+ 1613us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:03.243214 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:03.243416 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:03.244194 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:03.244197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:03.244216 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:03.244352 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.244377 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.246592 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.246596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.247867 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.247878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.250041 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.250045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.250143 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.250166 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:39:03.250842 (+ 676us) client_negotiation.cc:240] Negotiation successful
0807 08:39:03.250945 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1530,"mutex_wait_us":2,"thread_start_us":127,"threads_started":1}
I20260807 08:39:03.253504 6377 tablet_service.cc:1467] Processing CreateTablet for tablet b8d30a263c8e471a947beaf804c441d9 (DEFAULT_TABLE table=test-table [id=1c4391a5a976460b902df41bd3f1d1d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:39:03.253573 6199 tablet_service.cc:1467] Processing CreateTablet for tablet b8d30a263c8e471a947beaf804c441d9 (DEFAULT_TABLE table=test-table [id=1c4391a5a976460b902df41bd3f1d1d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:39:03.254313 6570 tablet_service.cc:1467] Processing CreateTablet for tablet b8d30a263c8e471a947beaf804c441d9 (DEFAULT_TABLE table=test-table [id=1c4391a5a976460b902df41bd3f1d1d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:39:03.255033 6199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8d30a263c8e471a947beaf804c441d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:03.255028 6377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8d30a263c8e471a947beaf804c441d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:03.255880 6570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8d30a263c8e471a947beaf804c441d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:03.263468 6678 tablet_bootstrap.cc:492] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a: Bootstrap starting.
I20260807 08:39:03.264878 6680 tablet_bootstrap.cc:492] T b8d30a263c8e471a947beaf804c441d9 P 0f11bf2498e34ce4ab9382ea2dfb4f09: Bootstrap starting.
I20260807 08:39:03.265373 6684 tablet_bootstrap.cc:492] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8: Bootstrap starting.
I20260807 08:39:03.267437 6678 tablet_bootstrap.cc:654] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:03.267859 6684 tablet_bootstrap.cc:654] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:03.268033 6680 tablet_bootstrap.cc:654] T b8d30a263c8e471a947beaf804c441d9 P 0f11bf2498e34ce4ab9382ea2dfb4f09: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:03.268721 6678 log.cc:824] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:03.268903 6680 log.cc:824] T b8d30a263c8e471a947beaf804c441d9 P 0f11bf2498e34ce4ab9382ea2dfb4f09: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:03.269227 6684 log.cc:824] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:03.271361 6678 tablet_bootstrap.cc:492] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a: No bootstrap required, opened a new log
I20260807 08:39:03.271385 6680 tablet_bootstrap.cc:492] T b8d30a263c8e471a947beaf804c441d9 P 0f11bf2498e34ce4ab9382ea2dfb4f09: No bootstrap required, opened a new log
I20260807 08:39:03.271384 6684 tablet_bootstrap.cc:492] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8: No bootstrap required, opened a new log
I20260807 08:39:03.271607 6678 ts_tablet_manager.cc:1397] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 08:39:03.271610 6680 ts_tablet_manager.cc:1397] T b8d30a263c8e471a947beaf804c441d9 P 0f11bf2498e34ce4ab9382ea2dfb4f09: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 08:39:03.271610 6684 ts_tablet_manager.cc:1397] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 08:39:03.276028 6678 raft_consensus.cc:348] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e5227d6f5045cd8000a368ad712b5a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 39813 } } peers { permanent_uuid: "1970aa9a90b24fb9a366455c0a4e00f8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 34149 } } peers { permanent_uuid: "0f11bf2498e34ce4ab9382ea2dfb4f09" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 37131 } }
I20260807 08:39:03.276396 6678 raft_consensus.cc:374] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:03.276456 6678 raft_consensus.cc:729] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37e5227d6f5045cd8000a368ad712b5a, State: Initialized, Role: FOLLOWER
I20260807 08:39:03.276568 6684 raft_consensus.cc:348] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e5227d6f5045cd8000a368ad712b5a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 39813 } } peers { permanent_uuid: "1970aa9a90b24fb9a366455c0a4e00f8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 34149 } } peers { permanent_uuid: "0f11bf2498e34ce4ab9382ea2dfb4f09" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 37131 } }
I20260807 08:39:03.276908 6684 raft_consensus.cc:374] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:03.276998 6684 raft_consensus.cc:729] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1970aa9a90b24fb9a366455c0a4e00f8, State: Initialized, Role: FOLLOWER
I20260807 08:39:03.277076 6678 consensus_queue.cc:260] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e5227d6f5045cd8000a368ad712b5a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 39813 } } peers { permanent_uuid: "1970aa9a90b24fb9a366455c0a4e00f8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 34149 } } peers { permanent_uuid: "0f11bf2498e34ce4ab9382ea2dfb4f09" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 37131 } }
I20260807 08:39:03.277510 6684 consensus_queue.cc:260] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e5227d6f5045cd8000a368ad712b5a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 39813 } } peers { permanent_uuid: "1970aa9a90b24fb9a366455c0a4e00f8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 34149 } } peers { permanent_uuid: "0f11bf2498e34ce4ab9382ea2dfb4f09" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 37131 } }
I20260807 08:39:03.277401 6680 raft_consensus.cc:348] T b8d30a263c8e471a947beaf804c441d9 P 0f11bf2498e34ce4ab9382ea2dfb4f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e5227d6f5045cd8000a368ad712b5a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 39813 } } peers { permanent_uuid: "1970aa9a90b24fb9a366455c0a4e00f8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 34149 } } peers { permanent_uuid: "0f11bf2498e34ce4ab9382ea2dfb4f09" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 37131 } }
I20260807 08:39:03.277674 6680 raft_consensus.cc:374] T b8d30a263c8e471a947beaf804c441d9 P 0f11bf2498e34ce4ab9382ea2dfb4f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:03.277745 6680 raft_consensus.cc:729] T b8d30a263c8e471a947beaf804c441d9 P 0f11bf2498e34ce4ab9382ea2dfb4f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f11bf2498e34ce4ab9382ea2dfb4f09, State: Initialized, Role: FOLLOWER
I20260807 08:39:03.278159 6479 heartbeater.cc:503] Master 127.9.174.62:40959 was elected leader, sending a full tablet report...
I20260807 08:39:03.278277 6680 consensus_queue.cc:260] T b8d30a263c8e471a947beaf804c441d9 P 0f11bf2498e34ce4ab9382ea2dfb4f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e5227d6f5045cd8000a368ad712b5a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 39813 } } peers { permanent_uuid: "1970aa9a90b24fb9a366455c0a4e00f8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 34149 } } peers { permanent_uuid: "0f11bf2498e34ce4ab9382ea2dfb4f09" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 37131 } }
I20260807 08:39:03.278488 6678 ts_tablet_manager.cc:1428] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260807 08:39:03.278669 6684 ts_tablet_manager.cc:1428] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260807 08:39:03.279219 6658 heartbeater.cc:503] Master 127.9.174.62:40959 was elected leader, sending a full tablet report...
I20260807 08:39:03.279330 6291 heartbeater.cc:503] Master 127.9.174.62:40959 was elected leader, sending a full tablet report...
I20260807 08:39:03.279393 6680 ts_tablet_manager.cc:1428] T b8d30a263c8e471a947beaf804c441d9 P 0f11bf2498e34ce4ab9382ea2dfb4f09: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 08:39:03.289353 6688 raft_consensus.cc:482] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:39:03.289659 6688 raft_consensus.cc:504] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e5227d6f5045cd8000a368ad712b5a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 39813 } } peers { permanent_uuid: "1970aa9a90b24fb9a366455c0a4e00f8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 34149 } } peers { permanent_uuid: "0f11bf2498e34ce4ab9382ea2dfb4f09" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 37131 } }
I20260807 08:39:03.291317 6688 leader_election.cc:288] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1970aa9a90b24fb9a366455c0a4e00f8 (127.9.174.3:34149), 0f11bf2498e34ce4ab9382ea2dfb4f09 (127.9.174.1:37131)
I20260807 08:39:03.296204 6484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.291572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:34149 (local address 127.9.174.2:52919)
0807 08:39:03.291767 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:03.291785 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:03.292013 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:03.292427 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:03.292430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:03.292458 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:03.292619 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.292626 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.293500 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.293507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.294975 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.294988 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.295788 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.295791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.295891 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.295916 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 08:39:03.295984 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 08:39:03.296056 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":43}
I20260807 08:39:03.296294 6674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.291746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:52919 (local address 127.9.174.3:34149)
0807 08:39:03.292009 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.292015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.292037 (+ 22us) server_negotiation.cc:407] Connection header received
0807 08:39:03.292147 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.292150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.292229 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.292317 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:03.292762 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.293368 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.295149 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.295632 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.295779 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.295871 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.296028 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.296129 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 08:39:03.297065 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.291970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:37131 (local address 127.9.174.2:42851)
0807 08:39:03.292445 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:03.292459 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:03.292614 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:03.292915 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:03.292918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:03.292936 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:03.293080 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.293087 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.294424 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.294427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.295591 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.295601 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.296685 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.296688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.296774 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.296795 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 08:39:03.296882 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 08:39:03.296941 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":4,"thread_start_us":187,"threads_started":1}
I20260807 08:39:03.297279 6599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8d30a263c8e471a947beaf804c441d9" candidate_uuid: "37e5227d6f5045cd8000a368ad712b5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1970aa9a90b24fb9a366455c0a4e00f8" is_pre_election: true
I20260807 08:39:03.297564 6675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.292136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:42851 (local address 127.9.174.1:37131)
0807 08:39:03.292341 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.292346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.292559 (+ 213us) server_negotiation.cc:407] Connection header received
0807 08:39:03.292741 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.292745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.292810 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.292905 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:03.293213 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.293924 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.295730 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.296396 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.296523 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.296612 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.297288 (+ 676us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.297401 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 08:39:03.297780 6599 raft_consensus.cc:2454] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37e5227d6f5045cd8000a368ad712b5a in term 0.
I20260807 08:39:03.298405 6322 leader_election.cc:302] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1970aa9a90b24fb9a366455c0a4e00f8, 37e5227d6f5045cd8000a368ad712b5a; no voters:
I20260807 08:39:03.298424 6225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8d30a263c8e471a947beaf804c441d9" candidate_uuid: "37e5227d6f5045cd8000a368ad712b5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f11bf2498e34ce4ab9382ea2dfb4f09" is_pre_election: true
I20260807 08:39:03.298681 6225 raft_consensus.cc:2454] T b8d30a263c8e471a947beaf804c441d9 P 0f11bf2498e34ce4ab9382ea2dfb4f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37e5227d6f5045cd8000a368ad712b5a in term 0.
I20260807 08:39:03.298750 6688 raft_consensus.cc:2781] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:39:03.298907 6688 raft_consensus.cc:482] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:39:03.298988 6688 raft_consensus.cc:3037] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:03.300536 6688 raft_consensus.cc:504] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e5227d6f5045cd8000a368ad712b5a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 39813 } } peers { permanent_uuid: "1970aa9a90b24fb9a366455c0a4e00f8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 34149 } } peers { permanent_uuid: "0f11bf2498e34ce4ab9382ea2dfb4f09" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 37131 } }
I20260807 08:39:03.301021 6688 leader_election.cc:288] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [CANDIDATE]: Term 1 election: Requested vote from peers 1970aa9a90b24fb9a366455c0a4e00f8 (127.9.174.3:34149), 0f11bf2498e34ce4ab9382ea2dfb4f09 (127.9.174.1:37131)
I20260807 08:39:03.301529 6225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8d30a263c8e471a947beaf804c441d9" candidate_uuid: "37e5227d6f5045cd8000a368ad712b5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f11bf2498e34ce4ab9382ea2dfb4f09"
I20260807 08:39:03.301553 6599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8d30a263c8e471a947beaf804c441d9" candidate_uuid: "37e5227d6f5045cd8000a368ad712b5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1970aa9a90b24fb9a366455c0a4e00f8"
I20260807 08:39:03.301752 6599 raft_consensus.cc:3037] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:03.301723 6225 raft_consensus.cc:3037] T b8d30a263c8e471a947beaf804c441d9 P 0f11bf2498e34ce4ab9382ea2dfb4f09 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:03.303488 6225 raft_consensus.cc:2454] T b8d30a263c8e471a947beaf804c441d9 P 0f11bf2498e34ce4ab9382ea2dfb4f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37e5227d6f5045cd8000a368ad712b5a in term 1.
I20260807 08:39:03.303496 6599 raft_consensus.cc:2454] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37e5227d6f5045cd8000a368ad712b5a in term 1.
I20260807 08:39:03.303841 6322 leader_election.cc:302] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f11bf2498e34ce4ab9382ea2dfb4f09, 37e5227d6f5045cd8000a368ad712b5a; no voters:
I20260807 08:39:03.304051 6688 raft_consensus.cc:2781] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:03.304356 6688 raft_consensus.cc:686] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [term 1 LEADER]: Becoming Leader. State: Replica: 37e5227d6f5045cd8000a368ad712b5a, State: Running, Role: LEADER
I20260807 08:39:03.304841 6688 consensus_queue.cc:237] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e5227d6f5045cd8000a368ad712b5a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 39813 } } peers { permanent_uuid: "1970aa9a90b24fb9a366455c0a4e00f8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 34149 } } peers { permanent_uuid: "0f11bf2498e34ce4ab9382ea2dfb4f09" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 37131 } }
I20260807 08:39:03.309250 6010 catalog_manager.cc:5495] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 37e5227d6f5045cd8000a368ad712b5a (127.9.174.2). New cstate: current_term: 1 leader_uuid: "37e5227d6f5045cd8000a368ad712b5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e5227d6f5045cd8000a368ad712b5a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 39813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1970aa9a90b24fb9a366455c0a4e00f8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 34149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f11bf2498e34ce4ab9382ea2dfb4f09" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 37131 } health_report { overall_health: UNKNOWN } } }
I20260807 08:39:03.324338 6671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.319987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40366 (local address 127.9.174.2:39813)
0807 08:39:03.320158 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.320163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.320180 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:39:03.320278 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.320280 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.320352 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.320465 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:39:03.320808 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.321394 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.322357 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.322601 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.322694 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.323129 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:39:03.323722 (+ 593us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:39:03.323833 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.324010 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.324127 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 08:39:03.332633 6599 raft_consensus.cc:1261] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8 [term 1 FOLLOWER]: Refusing update from remote peer 37e5227d6f5045cd8000a368ad712b5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:39:03.333144 6225 raft_consensus.cc:1261] T b8d30a263c8e471a947beaf804c441d9 P 0f11bf2498e34ce4ab9382ea2dfb4f09 [term 1 FOLLOWER]: Refusing update from remote peer 37e5227d6f5045cd8000a368ad712b5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:39:03.333678 6693 consensus_queue.cc:1035] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1970aa9a90b24fb9a366455c0a4e00f8" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 34149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:39:03.333938 6688 consensus_queue.cc:1035] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f11bf2498e34ce4ab9382ea2dfb4f09" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 37131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:39:03.343598 6702 mvcc.cc:205] Tried to move back new op lower bound from 7315832599880040448 to 7315832599780245504. Current Snapshot: MvccSnapshot[applied={T|T < 7315832599880040448}]
I20260807 08:39:03.354539 6011 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:39:03.354825 6011 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:39:03.358273 6011 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7734b06cbc5d43eab2dade27980ccdc7: Sending DeleteTablet for 3 replicas of tablet b8d30a263c8e471a947beaf804c441d9
I20260807 08:39:03.359342 6377 tablet_service.cc:1514] Processing DeleteTablet for tablet b8d30a263c8e471a947beaf804c441d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:39:03 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:40350
I20260807 08:39:03.359457 6570 tablet_service.cc:1514] Processing DeleteTablet for tablet b8d30a263c8e471a947beaf804c441d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:39:03 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:40972
I20260807 08:39:03.360090 6714 tablet_replica.cc:330] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a: stopping tablet replica
I20260807 08:39:03.360152 6199 tablet_service.cc:1514] Processing DeleteTablet for tablet b8d30a263c8e471a947beaf804c441d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:39:03 UTC) from {username='kudu', principal='kudu/127.9.174.62@KRBTEST.COM'} at 127.0.0.1:41136
I20260807 08:39:03.360710 6714 raft_consensus.cc:2229] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:39:03.360771 6716 tablet_replica.cc:330] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8: stopping tablet replica
I20260807 08:39:03.360805 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 6111
I20260807 08:39:03.361116 6716 raft_consensus.cc:2229] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:39:03.361531 6716 raft_consensus.cc:2258] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:39:03.365222 6714 raft_consensus.cc:2258] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:39:03.366185 6716 ts_tablet_manager.cc:1905] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:39:03.370044 6714 ts_tablet_manager.cc:1905] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:39:03.370055 6716 ts_tablet_manager.cc:1918] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:39:03.370185 6716 log.cc:1196] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/b8d30a263c8e471a947beaf804c441d9
I20260807 08:39:03.370613 6716 ts_tablet_manager.cc:1939] T b8d30a263c8e471a947beaf804c441d9 P 1970aa9a90b24fb9a366455c0a4e00f8: Deleting consensus metadata
I20260807 08:39:03.372637 5994 catalog_manager.cc:4841] TS 1970aa9a90b24fb9a366455c0a4e00f8 (127.9.174.3:34149): tablet b8d30a263c8e471a947beaf804c441d9 (table test-table [id=1c4391a5a976460b902df41bd3f1d1d6]) successfully deleted
I20260807 08:39:03.374392 6714 ts_tablet_manager.cc:1918] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:39:03.374554 6714 log.cc:1196] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/b8d30a263c8e471a947beaf804c441d9
I20260807 08:39:03.374943 6714 ts_tablet_manager.cc:1939] T b8d30a263c8e471a947beaf804c441d9 P 37e5227d6f5045cd8000a368ad712b5a: Deleting consensus metadata
I20260807 08:39:03.376777 5994 catalog_manager.cc:4841] TS 37e5227d6f5045cd8000a368ad712b5a (127.9.174.2:39813): tablet b8d30a263c8e471a947beaf804c441d9 (table test-table [id=1c4391a5a976460b902df41bd3f1d1d6]) successfully deleted
I20260807 08:39:03.392917 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 6297
W20260807 08:39:03.392860 5994 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:37131 (error 108)
I20260807 08:39:03.396678 6673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.394351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:37131 (local address 127.0.0.1:41148)
0807 08:39:03.396410 (+ 2059us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:03.396520 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1908}
W20260807 08:39:03.399328 5994 catalog_manager.cc:4584] TS 0f11bf2498e34ce4ab9382ea2dfb4f09 (127.9.174.1:37131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b8d30a263c8e471a947beaf804c441d9: Network error: Client connection negotiation failed: client connection to 127.9.174.1:37131: connect: Connection refused (error 111)
I20260807 08:39:03.409608 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 6488
I20260807 08:39:03.426685 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 5968
2026-08-07T08:39:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2854 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T08:39:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:03Z Disabled control of system clock
I20260807 08:39:03.467702 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:42141
--webserver_interface=127.9.174.62
--webserver_port=0
--builtin_ntp_servers=127.9.174.20:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:42141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:39:03.604589 6722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:03.604975 6722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:03.605016 6722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:03.609333 6722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:03.609429 6722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:03.609445 6722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:03.609463 6722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:03.609479 6722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:03.614754 6722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:42141
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:42141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:03.616359 6722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:03.618144 6722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:03.628304 6733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:03.628404 6735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:03.628424 6731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:03.628835 6722 server_base.cc:1034] running on GCE node
I20260807 08:39:03.629506 6722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:03.630904 6722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:03.632151 6722 hybrid_clock.cc:648] HybridClock initialized: now 1786091943632139 us; error 38 us; skew 500 ppm
I20260807 08:39:03.634572 6722 webserver.cc:460] Webserver started at http://127.9.174.62:36253/ using document root <none> and password file <none>
I20260807 08:39:03.635367 6722 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:03.635466 6722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:03.635758 6722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:03.638254 6722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "3be9af00773a441089818431234eb3ef"
format_stamp: "Formatted at 2026-08-07 08:39:03 on dist-test-slave-h3ft"
server_key: "c960aae56fd6439f0144b59b9cc43a30"
server_key_iv: "c1755bdab4eb9cf562b8af925408a860"
server_key_version: "encryptionkey@0"
I20260807 08:39:03.638917 6722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "3be9af00773a441089818431234eb3ef"
format_stamp: "Formatted at 2026-08-07 08:39:03 on dist-test-slave-h3ft"
server_key: "c960aae56fd6439f0144b59b9cc43a30"
server_key_iv: "c1755bdab4eb9cf562b8af925408a860"
server_key_version: "encryptionkey@0"
I20260807 08:39:03.643581 6722 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 08:39:03.646646 6742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:03.648380 6722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:39:03.648555 6722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "3be9af00773a441089818431234eb3ef"
format_stamp: "Formatted at 2026-08-07 08:39:03 on dist-test-slave-h3ft"
server_key: "c960aae56fd6439f0144b59b9cc43a30"
server_key_iv: "c1755bdab4eb9cf562b8af925408a860"
server_key_version: "encryptionkey@0"
I20260807 08:39:03.648701 6722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:03.674264 6722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:03.675343 6722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:03.675611 6722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:03.684528 6722 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:42141
I20260807 08:39:03.684551 6809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:42141 every 8 connection(s)
I20260807 08:39:03.685942 6722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:03.686419 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 6722
I20260807 08:39:03.686582 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:03.687089 9912 external_mini_cluster.cc:1428] Setting key e34a80cf45fc69b52b6e9fb1b6ee101a
I20260807 08:39:03.691135 6810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:03.700241 6810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef: Bootstrap starting.
I20260807 08:39:03.700508 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.688996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47130 (local address 127.9.174.62:42141)
0807 08:39:03.689943 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.689968 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.690023 (+ 55us) server_negotiation.cc:407] Connection header received
0807 08:39:03.691149 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.691181 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.691807 (+ 626us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.692289 (+ 482us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:03.692852 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.694479 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.695702 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.696454 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.696656 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.697815 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:03.697823 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:03.697843 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:03.698470 (+ 627us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:03.698583 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:03.698616 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:03.698733 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.698906 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.699292 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":378,"thread_start_us":164,"threads_started":1}
I20260807 08:39:03.703220 6810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:03.704326 6810 log.cc:824] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:03.708649 6810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef: No bootstrap required, opened a new log
I20260807 08:39:03.712984 6810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9af00773a441089818431234eb3ef" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42141 } }
I20260807 08:39:03.713296 6810 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:03.713337 6810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3be9af00773a441089818431234eb3ef, State: Initialized, Role: FOLLOWER
I20260807 08:39:03.713793 6810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9af00773a441089818431234eb3ef" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42141 } }
I20260807 08:39:03.713922 6810 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:03.713973 6810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:03.714062 6810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:03.715282 6810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9af00773a441089818431234eb3ef" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42141 } }
I20260807 08:39:03.715670 6810 leader_election.cc:302] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3be9af00773a441089818431234eb3ef; no voters:
I20260807 08:39:03.716082 6810 leader_election.cc:288] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:03.716210 6817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:03.716498 6817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [term 1 LEADER]: Becoming Leader. State: Replica: 3be9af00773a441089818431234eb3ef, State: Running, Role: LEADER
I20260807 08:39:03.716892 6817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9af00773a441089818431234eb3ef" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42141 } }
I20260807 08:39:03.717665 6810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:03.719043 6821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3be9af00773a441089818431234eb3ef. Latest consensus state: current_term: 1 leader_uuid: "3be9af00773a441089818431234eb3ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9af00773a441089818431234eb3ef" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42141 } } }
I20260807 08:39:03.719208 6821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:03.719676 6832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:03.721232 6819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3be9af00773a441089818431234eb3ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9af00773a441089818431234eb3ef" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 42141 } } }
I20260807 08:39:03.721381 6819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:03.724175 6832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:03.731809 6832 catalog_manager.cc:1347] Generated new cluster ID: 9cd63b840ac640aeab3cb6ef55452b98
I20260807 08:39:03.731931 6832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:03.746824 6832 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:03.748167 6832 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:03.755209 6832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef: Generated new TSK 0
I20260807 08:39:03.756213 6832 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:03.766186 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.1:0
--local_ip_for_outbound_sockets=127.9.174.1
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42141
--builtin_ntp_servers=127.9.174.20:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:39:03.922207 6848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:03.922592 6848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:03.922640 6848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:03.927393 6848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:03.927479 6848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:03.927552 6848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.1
I20260807 08:39:03.933517 6848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.174.1
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.1
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:03.935295 6848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:03.937561 6848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:03.950443 6856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:03.950443 6857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:03.950443 6859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:03.950912 6848 server_base.cc:1034] running on GCE node
I20260807 08:39:03.951575 6848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:03.952454 6848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:03.953666 6848 hybrid_clock.cc:648] HybridClock initialized: now 1786091943953612 us; error 134 us; skew 500 ppm
I20260807 08:39:03.956393 6848 webserver.cc:460] Webserver started at http://127.9.174.1:36623/ using document root <none> and password file <none>
I20260807 08:39:03.957252 6848 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:03.957362 6848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:03.957671 6848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:03.960654 6848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/data/instance:
uuid: "9114b9c08aa1469aaeaf5b0c47fd8472"
format_stamp: "Formatted at 2026-08-07 08:39:03 on dist-test-slave-h3ft"
server_key: "37ade8e42be8b544b9a29fccb1c7b314"
server_key_iv: "70e72344b4187543992cd3a59570258b"
server_key_version: "encryptionkey@0"
I20260807 08:39:03.961455 6848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance:
uuid: "9114b9c08aa1469aaeaf5b0c47fd8472"
format_stamp: "Formatted at 2026-08-07 08:39:03 on dist-test-slave-h3ft"
server_key: "37ade8e42be8b544b9a29fccb1c7b314"
server_key_iv: "70e72344b4187543992cd3a59570258b"
server_key_version: "encryptionkey@0"
I20260807 08:39:03.966619 6848 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:39:03.970074 6867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:03.971768 6848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 08:39:03.971940 6848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/wal
uuid: "9114b9c08aa1469aaeaf5b0c47fd8472"
format_stamp: "Formatted at 2026-08-07 08:39:03 on dist-test-slave-h3ft"
server_key: "37ade8e42be8b544b9a29fccb1c7b314"
server_key_iv: "70e72344b4187543992cd3a59570258b"
server_key_version: "encryptionkey@0"
I20260807 08:39:03.972116 6848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:03.988190 6848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:03.989248 6848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:03.989513 6848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:03.990353 6848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:03.991684 6848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:03.991767 6848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:03.991849 6848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:03.991896 6848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:04.004289 6848 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.1:43699
I20260807 08:39:04.004400 7018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.1:43699 every 8 connection(s)
I20260807 08:39:04.005633 6848 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/data/info.pb
I20260807 08:39:04.008361 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 6848
I20260807 08:39:04.008549 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-0/wal/instance
I20260807 08:39:04.008917 9912 external_mini_cluster.cc:1428] Setting key 1d87c2ce01c29f6e9388b5e69bed993e
I20260807 08:39:04.012003 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.2:0
--local_ip_for_outbound_sockets=127.9.174.2
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42141
--builtin_ntp_servers=127.9.174.20:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:39:04.019239 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.008345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.1:55645 (local address 127.9.174.62:42141)
0807 08:39:04.008563 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:04.008569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:04.010077 (+ 1508us) server_negotiation.cc:407] Connection header received
0807 08:39:04.012018 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:04.012025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:04.012106 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:04.012212 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:04.013939 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.014972 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.016503 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.016750 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.016882 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.018445 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:04.018448 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:04.018451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:04.018468 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:04.018543 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:04.018556 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:04.018647 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:04.018847 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 08:39:04.018983 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 08:39:04.020553 7025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.008902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42141 (local address 127.9.174.1:55645)
0807 08:39:04.009881 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:04.009948 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:04.011762 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:04.012631 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:04.012644 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:04.012779 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:04.013707 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:04.013733 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.015196 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.015200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:04.016328 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:04.016341 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.016961 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.016967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:04.017215 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.018081 (+ 866us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:39:04.018083 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:04.018142 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:39:04.018240 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:04.018650 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:04.018664 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:04.018675 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 08:39:04.018849 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 08:39:04.019276 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":175,"threads_started":1}
I20260807 08:39:04.022264 7022 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42141
I20260807 08:39:04.022619 7022 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:04.023408 7022 heartbeater.cc:511] Master 127.9.174.62:42141 requested a full tablet report, sending...
I20260807 08:39:04.025676 6763 ts_manager.cc:194] Registered new tserver with Master: 9114b9c08aa1469aaeaf5b0c47fd8472 (127.9.174.1:43699)
I20260807 08:39:04.028012 6763 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.174.1:55645
W20260807 08:39:04.164707 7026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:04.165113 7026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:04.165165 7026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:04.169857 7026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:04.169942 7026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:04.170012 7026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.2
I20260807 08:39:04.175315 7026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.174.2
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.2
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:04.176920 7026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:04.178814 7026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:04.191375 7034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:04.191336 7033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:04.191336 7037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:04.192001 7026 server_base.cc:1034] running on GCE node
I20260807 08:39:04.192564 7026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:04.193536 7026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:04.194769 7026 hybrid_clock.cc:648] HybridClock initialized: now 1786091944194742 us; error 42 us; skew 500 ppm
I20260807 08:39:04.197809 7026 webserver.cc:460] Webserver started at http://127.9.174.2:43539/ using document root <none> and password file <none>
I20260807 08:39:04.198604 7026 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:04.198711 7026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:04.198989 7026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:04.201779 7026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/data/instance:
uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a"
format_stamp: "Formatted at 2026-08-07 08:39:04 on dist-test-slave-h3ft"
server_key: "cc33ddd6d019478d303a781505c622da"
server_key_iv: "f666ff0d686b2fb0d1ed6ca020d39eb6"
server_key_version: "encryptionkey@0"
I20260807 08:39:04.202518 7026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance:
uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a"
format_stamp: "Formatted at 2026-08-07 08:39:04 on dist-test-slave-h3ft"
server_key: "cc33ddd6d019478d303a781505c622da"
server_key_iv: "f666ff0d686b2fb0d1ed6ca020d39eb6"
server_key_version: "encryptionkey@0"
I20260807 08:39:04.207576 7026 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 08:39:04.210608 7046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:04.212191 7026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:39:04.212360 7026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/wal
uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a"
format_stamp: "Formatted at 2026-08-07 08:39:04 on dist-test-slave-h3ft"
server_key: "cc33ddd6d019478d303a781505c622da"
server_key_iv: "f666ff0d686b2fb0d1ed6ca020d39eb6"
server_key_version: "encryptionkey@0"
I20260807 08:39:04.212507 7026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:04.239355 7026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:04.240384 7026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:04.240651 7026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:04.241478 7026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:04.242775 7026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:04.242856 7026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:04.242959 7026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:04.243026 7026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260807 08:39:04.254690 7026 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.2:35607
I20260807 08:39:04.254715 7204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.2:35607 every 8 connection(s)
I20260807 08:39:04.256018 7026 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/data/info.pb
I20260807 08:39:04.264062 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 7026
I20260807 08:39:04.264194 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/wal/instance
I20260807 08:39:04.264592 9912 external_mini_cluster.cc:1428] Setting key e619f7fcfa336da71a10523f2fec08f0
I20260807 08:39:04.267283 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.258433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:54185 (local address 127.9.174.62:42141)
0807 08:39:04.258596 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:04.258600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:04.259966 (+ 1366us) server_negotiation.cc:407] Connection header received
0807 08:39:04.261643 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:04.261647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:04.261726 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:04.261831 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:04.262888 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.263543 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.264984 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.265233 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.265351 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.266675 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:04.266679 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:04.266681 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:04.266696 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:04.266726 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:04.266734 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:04.266811 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:04.266985 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 08:39:04.267104 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:39:04.267247 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.174.3:0
--local_ip_for_outbound_sockets=127.9.174.3
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42141
--builtin_ntp_servers=127.9.174.20:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:39:04.268452 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.258929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42141 (local address 127.9.174.2:54185)
0807 08:39:04.259794 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:04.259840 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:04.261386 (+ 1546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:04.262028 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:04.262044 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:04.262142 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:04.262733 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:04.262749 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.263701 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.263705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:04.264852 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:04.264864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.265412 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.265417 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:04.265609 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.266372 (+ 763us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:39:04.266374 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:04.266425 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:39:04.266505 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:04.266825 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:04.266842 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:04.266847 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:39:04.266985 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 08:39:04.267378 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":159,"threads_started":1}
I20260807 08:39:04.269829 7205 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42141
I20260807 08:39:04.270171 7205 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:04.271024 7205 heartbeater.cc:511] Master 127.9.174.62:42141 requested a full tablet report, sending...
I20260807 08:39:04.272571 6763 ts_manager.cc:194] Registered new tserver with Master: 4e0b87de277d4e0ab4e49e001b8f0c9a (127.9.174.2:35607)
I20260807 08:39:04.273284 6763 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.174.2:54185
W20260807 08:39:04.409844 7210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:04.410198 7210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:04.410243 7210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:04.414634 7210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:04.414724 7210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:04.414786 7210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.174.3
I20260807 08:39:04.420421 7210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.174.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.174.3
--webserver_port=0
--tserver_master_addrs=127.9.174.62:42141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.174.3
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:04.421918 7210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:04.423801 7210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:04.435911 7216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:04.436021 7220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:04.435930 7217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:04.436267 7210 server_base.cc:1034] running on GCE node
I20260807 08:39:04.436826 7210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:04.437760 7210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:04.438958 7210 hybrid_clock.cc:648] HybridClock initialized: now 1786091944438928 us; error 45 us; skew 500 ppm
I20260807 08:39:04.441319 7210 webserver.cc:460] Webserver started at http://127.9.174.3:36013/ using document root <none> and password file <none>
I20260807 08:39:04.442054 7210 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:04.442135 7210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:04.442415 7210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:04.445099 7210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/data/instance:
uuid: "c89bbc0ccaa34e97954ff1d97c7dfd42"
format_stamp: "Formatted at 2026-08-07 08:39:04 on dist-test-slave-h3ft"
server_key: "986d162d90e914cad34efafdf6335a98"
server_key_iv: "b657d0afee9804ab7f0b1fca41e154b9"
server_key_version: "encryptionkey@0"
I20260807 08:39:04.445811 7210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance:
uuid: "c89bbc0ccaa34e97954ff1d97c7dfd42"
format_stamp: "Formatted at 2026-08-07 08:39:04 on dist-test-slave-h3ft"
server_key: "986d162d90e914cad34efafdf6335a98"
server_key_iv: "b657d0afee9804ab7f0b1fca41e154b9"
server_key_version: "encryptionkey@0"
I20260807 08:39:04.450551 7210 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260807 08:39:04.454109 7229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:04.456059 7210 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:39:04.456245 7210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/wal
uuid: "c89bbc0ccaa34e97954ff1d97c7dfd42"
format_stamp: "Formatted at 2026-08-07 08:39:04 on dist-test-slave-h3ft"
server_key: "986d162d90e914cad34efafdf6335a98"
server_key_iv: "b657d0afee9804ab7f0b1fca41e154b9"
server_key_version: "encryptionkey@0"
I20260807 08:39:04.456408 7210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:04.470702 7210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:04.471782 7210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:04.472049 7210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:04.472892 7210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:04.474166 7210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:04.474247 7210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:04.474334 7210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:04.474388 7210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:04.486886 7210 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.3:46675
I20260807 08:39:04.486888 7378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.3:46675 every 8 connection(s)
I20260807 08:39:04.488232 7210 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/data/info.pb
I20260807 08:39:04.488740 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 7210
I20260807 08:39:04.488912 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/wal/instance
I20260807 08:39:04.489329 9912 external_mini_cluster.cc:1428] Setting key b2473c07bac33ee0f964d0d7dc1970b2
I20260807 08:39:04.500062 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.491256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.3:43217 (local address 127.9.174.62:42141)
0807 08:39:04.491425 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:04.491429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:04.492951 (+ 1522us) server_negotiation.cc:407] Connection header received
0807 08:39:04.494642 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:04.494646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:04.494716 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:04.494800 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:04.495916 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.496516 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.497796 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.498039 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.498150 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.499482 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:04.499486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:04.499489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:04.499504 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:04.499532 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:04.499539 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:04.499614 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:04.499793 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 08:39:04.499871 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 08:39:04.501158 7384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.491706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:42141 (local address 127.9.174.3:43217)
0807 08:39:04.492787 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:04.492837 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:04.494404 (+ 1567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:04.495051 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:04.495060 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:04.495159 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:04.495771 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:04.495785 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.496656 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.496659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:04.497660 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:04.497671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.498209 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.498212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:04.498396 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.499178 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:39:04.499180 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:04.499230 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:39:04.499307 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:04.499628 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:04.499644 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:04.499649 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:39:04.499778 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 08:39:04.500123 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":221,"threads_started":1}
I20260807 08:39:04.502616 7379 heartbeater.cc:347] Connected to a master server at 127.9.174.62:42141
I20260807 08:39:04.502965 7379 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:04.503768 7379 heartbeater.cc:511] Master 127.9.174.62:42141 requested a full tablet report, sending...
I20260807 08:39:04.504983 6763 ts_manager.cc:194] Registered new tserver with Master: c89bbc0ccaa34e97954ff1d97c7dfd42 (127.9.174.3:46675)
I20260807 08:39:04.505717 6763 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.174.3:43217
I20260807 08:39:04.516693 9912 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:39:04.524801 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.519841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47138 (local address 127.9.174.62:42141)
0807 08:39:04.520045 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:04.520049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:04.520391 (+ 342us) server_negotiation.cc:407] Connection header received
0807 08:39:04.520574 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:04.520578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:04.520644 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:04.520747 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:04.521124 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.521741 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.522753 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.523052 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.523184 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.524264 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:04.524268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:04.524270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:04.524283 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:04.524308 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:04.524317 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:04.524424 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:04.524522 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 08:39:04.524593 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 08:39:04.528715 6763 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:39:04.532912 6763 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:39:04.551770 7400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.545155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:43699 (local address 127.0.0.1:57448)
0807 08:39:04.545719 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:04.545778 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:04.545937 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:04.546598 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:04.546605 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:04.546625 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:04.546852 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:04.546860 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.548461 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.548464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:04.550098 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:04.550108 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.551260 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.551264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:04.551391 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.551434 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 08:39:04.551514 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 08:39:04.551617 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":118,"threads_started":1}
I20260807 08:39:04.552376 7398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.545242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:46675 (local address 127.0.0.1:36548)
0807 08:39:04.546516 (+ 1274us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:04.546533 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:04.546619 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:04.548362 (+ 1743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:04.548365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:04.548386 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:04.548511 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:04.548522 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.549535 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.549539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:04.550708 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:04.550720 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.551962 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.551966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:04.552063 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.552089 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 08:39:04.552140 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 08:39:04.552214 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1201,"mutex_wait_us":1,"thread_start_us":141,"threads_started":1}
I20260807 08:39:04.552927 7401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.545346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57448 (local address 127.9.174.1:43699)
0807 08:39:04.545836 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:04.545844 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:04.545878 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:39:04.546145 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:04.546157 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:04.546401 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:04.546579 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:04.547102 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.548316 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.550239 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.551113 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.552235 (+ 1122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.552527 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:04.552598 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 08:39:04.552719 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":107,"threads_started":1}
I20260807 08:39:04.553221 7403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.546526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.2:35607 (local address 127.0.0.1:56112)
0807 08:39:04.547182 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:04.547197 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:04.547299 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:04.548144 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:04.548148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:04.548167 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:04.548293 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:04.548299 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.549948 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.549951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:04.551181 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:04.551192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.552398 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.552402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:04.552713 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.552738 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 08:39:04.552820 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:39:04.552887 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":122,"threads_started":1}
I20260807 08:39:04.553243 7402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.546044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36548 (local address 127.9.174.3:46675)
0807 08:39:04.547644 (+ 1600us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:04.547661 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:04.547691 (+ 30us) server_negotiation.cc:407] Connection header received
0807 08:39:04.547992 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:04.547998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:04.548186 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:04.548377 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:04.548619 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.549398 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.550881 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.551800 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.552691 (+ 891us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.552879 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:04.552949 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 08:39:04.553046 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1027,"thread_start_us":194,"threads_started":1}
I20260807 08:39:04.553249 7404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.547062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56112 (local address 127.9.174.2:35607)
0807 08:39:04.547651 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:04.547666 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:04.547690 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:39:04.547787 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:04.547792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:04.547989 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:04.548147 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:04.548462 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.549818 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.551329 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.552269 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.552378 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.552631 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:04.552945 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 08:39:04.553049 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":141,"threads_started":1}
I20260807 08:39:04.556515 7114 tablet_service.cc:1467] Processing CreateTablet for tablet bcf0210ea44947a6ab39b8f1040129e7 (DEFAULT_TABLE table=test-table [id=15a28bbd259d48fbbd8a83163bbbc450]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:39:04.556542 6928 tablet_service.cc:1467] Processing CreateTablet for tablet bcf0210ea44947a6ab39b8f1040129e7 (DEFAULT_TABLE table=test-table [id=15a28bbd259d48fbbd8a83163bbbc450]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:39:04.556527 7295 tablet_service.cc:1467] Processing CreateTablet for tablet bcf0210ea44947a6ab39b8f1040129e7 (DEFAULT_TABLE table=test-table [id=15a28bbd259d48fbbd8a83163bbbc450]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:39:04.557801 7114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcf0210ea44947a6ab39b8f1040129e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:04.557804 7295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcf0210ea44947a6ab39b8f1040129e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:04.557804 6928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcf0210ea44947a6ab39b8f1040129e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:04.566097 7406 tablet_bootstrap.cc:492] T bcf0210ea44947a6ab39b8f1040129e7 P 9114b9c08aa1469aaeaf5b0c47fd8472: Bootstrap starting.
I20260807 08:39:04.567370 7408 tablet_bootstrap.cc:492] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42: Bootstrap starting.
I20260807 08:39:04.567432 7405 tablet_bootstrap.cc:492] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a: Bootstrap starting.
I20260807 08:39:04.569152 7406 tablet_bootstrap.cc:654] T bcf0210ea44947a6ab39b8f1040129e7 P 9114b9c08aa1469aaeaf5b0c47fd8472: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:04.569173 7408 tablet_bootstrap.cc:654] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:04.569533 7405 tablet_bootstrap.cc:654] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:04.570132 7408 log.cc:824] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:04.570150 7406 log.cc:824] T bcf0210ea44947a6ab39b8f1040129e7 P 9114b9c08aa1469aaeaf5b0c47fd8472: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:04.570338 7405 log.cc:824] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:04.572613 7408 tablet_bootstrap.cc:492] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42: No bootstrap required, opened a new log
I20260807 08:39:04.572628 7405 tablet_bootstrap.cc:492] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a: No bootstrap required, opened a new log
I20260807 08:39:04.572645 7406 tablet_bootstrap.cc:492] T bcf0210ea44947a6ab39b8f1040129e7 P 9114b9c08aa1469aaeaf5b0c47fd8472: No bootstrap required, opened a new log
I20260807 08:39:04.572844 7406 ts_tablet_manager.cc:1397] T bcf0210ea44947a6ab39b8f1040129e7 P 9114b9c08aa1469aaeaf5b0c47fd8472: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 08:39:04.572844 7405 ts_tablet_manager.cc:1397] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 08:39:04.572845 7408 ts_tablet_manager.cc:1397] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260807 08:39:04.576604 7408 raft_consensus.cc:348] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9114b9c08aa1469aaeaf5b0c47fd8472" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43699 } } peers { permanent_uuid: "c89bbc0ccaa34e97954ff1d97c7dfd42" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46675 } } peers { permanent_uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 35607 } }
I20260807 08:39:04.576927 7408 raft_consensus.cc:374] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:04.576987 7408 raft_consensus.cc:729] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c89bbc0ccaa34e97954ff1d97c7dfd42, State: Initialized, Role: FOLLOWER
I20260807 08:39:04.577528 7408 consensus_queue.cc:260] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9114b9c08aa1469aaeaf5b0c47fd8472" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43699 } } peers { permanent_uuid: "c89bbc0ccaa34e97954ff1d97c7dfd42" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46675 } } peers { permanent_uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 35607 } }
I20260807 08:39:04.577590 7405 raft_consensus.cc:348] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9114b9c08aa1469aaeaf5b0c47fd8472" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43699 } } peers { permanent_uuid: "c89bbc0ccaa34e97954ff1d97c7dfd42" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46675 } } peers { permanent_uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 35607 } }
I20260807 08:39:04.577975 7405 raft_consensus.cc:374] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:04.578043 7405 raft_consensus.cc:729] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0b87de277d4e0ab4e49e001b8f0c9a, State: Initialized, Role: FOLLOWER
I20260807 08:39:04.577981 7406 raft_consensus.cc:348] T bcf0210ea44947a6ab39b8f1040129e7 P 9114b9c08aa1469aaeaf5b0c47fd8472 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9114b9c08aa1469aaeaf5b0c47fd8472" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43699 } } peers { permanent_uuid: "c89bbc0ccaa34e97954ff1d97c7dfd42" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46675 } } peers { permanent_uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 35607 } }
I20260807 08:39:04.578233 7406 raft_consensus.cc:374] T bcf0210ea44947a6ab39b8f1040129e7 P 9114b9c08aa1469aaeaf5b0c47fd8472 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:04.578293 7406 raft_consensus.cc:729] T bcf0210ea44947a6ab39b8f1040129e7 P 9114b9c08aa1469aaeaf5b0c47fd8472 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9114b9c08aa1469aaeaf5b0c47fd8472, State: Initialized, Role: FOLLOWER
I20260807 08:39:04.578519 7405 consensus_queue.cc:260] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9114b9c08aa1469aaeaf5b0c47fd8472" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43699 } } peers { permanent_uuid: "c89bbc0ccaa34e97954ff1d97c7dfd42" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46675 } } peers { permanent_uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 35607 } }
I20260807 08:39:04.578644 7379 heartbeater.cc:503] Master 127.9.174.62:42141 was elected leader, sending a full tablet report...
I20260807 08:39:04.578824 7408 ts_tablet_manager.cc:1428] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 08:39:04.579363 7205 heartbeater.cc:503] Master 127.9.174.62:42141 was elected leader, sending a full tablet report...
I20260807 08:39:04.579614 7405 ts_tablet_manager.cc:1428] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 08:39:04.580188 7406 consensus_queue.cc:260] T bcf0210ea44947a6ab39b8f1040129e7 P 9114b9c08aa1469aaeaf5b0c47fd8472 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9114b9c08aa1469aaeaf5b0c47fd8472" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43699 } } peers { permanent_uuid: "c89bbc0ccaa34e97954ff1d97c7dfd42" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46675 } } peers { permanent_uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 35607 } }
I20260807 08:39:04.580935 7022 heartbeater.cc:503] Master 127.9.174.62:42141 was elected leader, sending a full tablet report...
I20260807 08:39:04.581212 7406 ts_tablet_manager.cc:1428] T bcf0210ea44947a6ab39b8f1040129e7 P 9114b9c08aa1469aaeaf5b0c47fd8472: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260807 08:39:04.696313 7418 raft_consensus.cc:482] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:39:04.696691 7418 raft_consensus.cc:504] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9114b9c08aa1469aaeaf5b0c47fd8472" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43699 } } peers { permanent_uuid: "c89bbc0ccaa34e97954ff1d97c7dfd42" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46675 } } peers { permanent_uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 35607 } }
I20260807 08:39:04.698423 7418 leader_election.cc:288] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9114b9c08aa1469aaeaf5b0c47fd8472 (127.9.174.1:43699), c89bbc0ccaa34e97954ff1d97c7dfd42 (127.9.174.3:46675)
I20260807 08:39:04.702546 7402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.698895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:55975 (local address 127.9.174.3:46675)
0807 08:39:04.699086 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:04.699091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:04.699107 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:39:04.699285 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:04.699288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:04.699339 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:04.699428 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:04.699767 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.700327 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.701431 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.701866 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.701989 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.702080 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:04.702301 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 08:39:04.702391 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 08:39:04.702545 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.698692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.3:46675 (local address 127.9.174.2:55975)
0807 08:39:04.698959 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:04.698979 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:04.699121 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:04.699446 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:04.699449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:04.699472 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:04.699640 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:04.699647 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.700444 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.700449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:04.701291 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:04.701300 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.702006 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.702010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:04.702168 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.702188 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 08:39:04.702279 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 08:39:04.702372 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":22}
I20260807 08:39:04.703470 7421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.698692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:43699 (local address 127.9.174.2:34865)
0807 08:39:04.699556 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:04.699569 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:04.699705 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:04.699994 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:04.699997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:04.700013 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:04.700158 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:04.700164 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.701114 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.701119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:04.702074 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:04.702084 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.702951 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.702955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:04.703097 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.703149 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 08:39:04.703233 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 08:39:04.703313 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"mutex_wait_us":455,"thread_start_us":191,"threads_started":1}
I20260807 08:39:04.703584 7401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.699024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.174.2:34865 (local address 127.9.174.1:43699)
0807 08:39:04.699204 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:04.699211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:04.699666 (+ 455us) server_negotiation.cc:407] Connection header received
0807 08:39:04.699828 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:04.699831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:04.699893 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:04.699981 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:04.700361 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.700966 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.702227 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.702794 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.702921 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.703139 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:04.703273 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 08:39:04.703397 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 08:39:04.703622 7318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcf0210ea44947a6ab39b8f1040129e7" candidate_uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c89bbc0ccaa34e97954ff1d97c7dfd42" is_pre_election: true
I20260807 08:39:04.704099 7318 raft_consensus.cc:2454] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0b87de277d4e0ab4e49e001b8f0c9a in term 0.
I20260807 08:39:04.704366 6953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcf0210ea44947a6ab39b8f1040129e7" candidate_uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9114b9c08aa1469aaeaf5b0c47fd8472" is_pre_election: true
I20260807 08:39:04.704718 6953 raft_consensus.cc:2454] T bcf0210ea44947a6ab39b8f1040129e7 P 9114b9c08aa1469aaeaf5b0c47fd8472 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0b87de277d4e0ab4e49e001b8f0c9a in term 0.
I20260807 08:39:04.704823 7051 leader_election.cc:302] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0b87de277d4e0ab4e49e001b8f0c9a, c89bbc0ccaa34e97954ff1d97c7dfd42; no voters:
I20260807 08:39:04.705206 7418 raft_consensus.cc:2781] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:39:04.705324 7418 raft_consensus.cc:482] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:39:04.705390 7418 raft_consensus.cc:3037] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:04.706769 7418 raft_consensus.cc:504] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9114b9c08aa1469aaeaf5b0c47fd8472" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43699 } } peers { permanent_uuid: "c89bbc0ccaa34e97954ff1d97c7dfd42" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46675 } } peers { permanent_uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 35607 } }
I20260807 08:39:04.707242 7418 leader_election.cc:288] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [CANDIDATE]: Term 1 election: Requested vote from peers 9114b9c08aa1469aaeaf5b0c47fd8472 (127.9.174.1:43699), c89bbc0ccaa34e97954ff1d97c7dfd42 (127.9.174.3:46675)
I20260807 08:39:04.707727 6953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcf0210ea44947a6ab39b8f1040129e7" candidate_uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9114b9c08aa1469aaeaf5b0c47fd8472"
I20260807 08:39:04.707747 7318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcf0210ea44947a6ab39b8f1040129e7" candidate_uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c89bbc0ccaa34e97954ff1d97c7dfd42"
I20260807 08:39:04.707893 6953 raft_consensus.cc:3037] T bcf0210ea44947a6ab39b8f1040129e7 P 9114b9c08aa1469aaeaf5b0c47fd8472 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:04.707918 7318 raft_consensus.cc:3037] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:04.709631 6953 raft_consensus.cc:2454] T bcf0210ea44947a6ab39b8f1040129e7 P 9114b9c08aa1469aaeaf5b0c47fd8472 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0b87de277d4e0ab4e49e001b8f0c9a in term 1.
I20260807 08:39:04.709640 7318 raft_consensus.cc:2454] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0b87de277d4e0ab4e49e001b8f0c9a in term 1.
I20260807 08:39:04.710306 7051 leader_election.cc:302] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0b87de277d4e0ab4e49e001b8f0c9a, c89bbc0ccaa34e97954ff1d97c7dfd42; no voters:
I20260807 08:39:04.710652 7418 raft_consensus.cc:2781] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:04.710912 7418 raft_consensus.cc:686] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [term 1 LEADER]: Becoming Leader. State: Replica: 4e0b87de277d4e0ab4e49e001b8f0c9a, State: Running, Role: LEADER
I20260807 08:39:04.711431 7418 consensus_queue.cc:237] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9114b9c08aa1469aaeaf5b0c47fd8472" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43699 } } peers { permanent_uuid: "c89bbc0ccaa34e97954ff1d97c7dfd42" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46675 } } peers { permanent_uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 35607 } }
I20260807 08:39:04.715804 6762 catalog_manager.cc:5495] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0b87de277d4e0ab4e49e001b8f0c9a (127.9.174.2). New cstate: current_term: 1 leader_uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9114b9c08aa1469aaeaf5b0c47fd8472" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c89bbc0ccaa34e97954ff1d97c7dfd42" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0b87de277d4e0ab4e49e001b8f0c9a" member_type: VOTER last_known_addr { host: "127.9.174.2" port: 35607 } health_report { overall_health: HEALTHY } } }
W20260807 08:39:04.739672 7382 tablet.cc:2369] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:39:04.745882 7404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.741407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56116 (local address 127.9.174.2:35607)
0807 08:39:04.741619 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:04.741624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:04.741644 (+ 20us) server_negotiation.cc:407] Connection header received
0807 08:39:04.741758 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:04.741761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:04.741845 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:04.741993 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:39:04.742372 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.742946 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.743966 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:04.744220 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:04.744314 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:04.744752 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:39:04.745359 (+ 607us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:39:04.745462 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:04.745596 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 08:39:04.745665 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 08:39:04.753758 7318 raft_consensus.cc:1261] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42 [term 1 FOLLOWER]: Refusing update from remote peer 4e0b87de277d4e0ab4e49e001b8f0c9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:39:04.753773 6953 raft_consensus.cc:1261] T bcf0210ea44947a6ab39b8f1040129e7 P 9114b9c08aa1469aaeaf5b0c47fd8472 [term 1 FOLLOWER]: Refusing update from remote peer 4e0b87de277d4e0ab4e49e001b8f0c9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:39:04.754674 7422 consensus_queue.cc:1035] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c89bbc0ccaa34e97954ff1d97c7dfd42" member_type: VOTER last_known_addr { host: "127.9.174.3" port: 46675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:39:04.755118 7418 consensus_queue.cc:1035] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9114b9c08aa1469aaeaf5b0c47fd8472" member_type: VOTER last_known_addr { host: "127.9.174.1" port: 43699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 08:39:04.757434 7206 tablet.cc:2369] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:39:04.757473 7024 tablet.cc:2369] T bcf0210ea44947a6ab39b8f1040129e7 P 9114b9c08aa1469aaeaf5b0c47fd8472: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:39:04.767750 7428 mvcc.cc:205] Tried to move back new op lower bound from 7315832605701550080 to 7315832605542486016. Current Snapshot: MvccSnapshot[applied={T|T < 7315832605701550080}]
I20260807 08:39:04.778601 6763 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:39:04.778991 6763 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:39:04.782339 6763 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3be9af00773a441089818431234eb3ef: Sending DeleteTablet for 3 replicas of tablet bcf0210ea44947a6ab39b8f1040129e7
I20260807 08:39:04.783535 7295 tablet_service.cc:1514] Processing DeleteTablet for tablet bcf0210ea44947a6ab39b8f1040129e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:39:04 UTC) from {username='slave'} at 127.0.0.1:36548
I20260807 08:39:04.783524 6928 tablet_service.cc:1514] Processing DeleteTablet for tablet bcf0210ea44947a6ab39b8f1040129e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:39:04 UTC) from {username='slave'} at 127.0.0.1:57448
I20260807 08:39:04.783895 7114 tablet_service.cc:1514] Processing DeleteTablet for tablet bcf0210ea44947a6ab39b8f1040129e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:39:04 UTC) from {username='slave'} at 127.0.0.1:56112
I20260807 08:39:04.784461 7440 tablet_replica.cc:330] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42: stopping tablet replica
I20260807 08:39:04.784950 7440 raft_consensus.cc:2229] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:39:04.785148 7441 tablet_replica.cc:330] T bcf0210ea44947a6ab39b8f1040129e7 P 9114b9c08aa1469aaeaf5b0c47fd8472: stopping tablet replica
I20260807 08:39:04.785396 7440 raft_consensus.cc:2258] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:39:04.785480 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 6848
I20260807 08:39:04.794565 7440 ts_tablet_manager.cc:1905] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:39:04.798661 7440 ts_tablet_manager.cc:1918] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:39:04.798820 7440 log.cc:1196] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-2/wal/wals/bcf0210ea44947a6ab39b8f1040129e7
I20260807 08:39:04.799286 7440 ts_tablet_manager.cc:1939] T bcf0210ea44947a6ab39b8f1040129e7 P c89bbc0ccaa34e97954ff1d97c7dfd42: Deleting consensus metadata
I20260807 08:39:04.801309 6745 catalog_manager.cc:4841] TS c89bbc0ccaa34e97954ff1d97c7dfd42 (127.9.174.3:46675): tablet bcf0210ea44947a6ab39b8f1040129e7 (table test-table [id=15a28bbd259d48fbbd8a83163bbbc450]) successfully deleted
I20260807 08:39:04.801658 7443 tablet_replica.cc:330] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a: stopping tablet replica
I20260807 08:39:04.802167 7443 raft_consensus.cc:2229] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:39:04.802783 7443 raft_consensus.cc:2258] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:39:04.805172 7443 ts_tablet_manager.cc:1905] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:39:04.808918 7443 ts_tablet_manager.cc:1918] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:39:04.809084 7443 log.cc:1196] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a: Deleting WAL directory at /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091655732880-9912-0/minicluster-data/ts-1/wal/wals/bcf0210ea44947a6ab39b8f1040129e7
I20260807 08:39:04.809463 7443 ts_tablet_manager.cc:1939] T bcf0210ea44947a6ab39b8f1040129e7 P 4e0b87de277d4e0ab4e49e001b8f0c9a: Deleting consensus metadata
W20260807 08:39:04.811154 6747 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.174.1:43699 (error 108)
I20260807 08:39:04.811235 6743 catalog_manager.cc:4841] TS 4e0b87de277d4e0ab4e49e001b8f0c9a (127.9.174.2:35607): tablet bcf0210ea44947a6ab39b8f1040129e7 (table test-table [id=15a28bbd259d48fbbd8a83163bbbc450]) successfully deleted
I20260807 08:39:04.812065 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 7026
I20260807 08:39:04.826860 7403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:04.812280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.1:43699 (local address 127.0.0.1:57460)
0807 08:39:04.826500 (+ 14220us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:04.826679 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.1:43699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14006}
W20260807 08:39:04.827464 6747 catalog_manager.cc:4584] TS 9114b9c08aa1469aaeaf5b0c47fd8472 (127.9.174.1:43699): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bcf0210ea44947a6ab39b8f1040129e7: Network error: Client connection negotiation failed: client connection to 127.9.174.1:43699: connect: Connection refused (error 111)
I20260807 08:39:04.841609 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 7210
I20260807 08:39:04.856730 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 6722
2026-08-07T08:39:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1436 ms)
[----------] 6 tests from EncryptionPolicyTest (12674 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:04 dist-test-slave-h3ft krb5kdc[7449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:04 dist-test-slave-h3ft krb5kdc[7449](info): set up 2 sockets
Aug 07 08:39:04 dist-test-slave-h3ft krb5kdc[7449](info): commencing operation
krb5kdc: starting...
W20260807 08:39:06.167937 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.004s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:06 dist-test-slave-h3ft krb5kdc[7449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:39:06.323467 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45599
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:06.456067 7466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:06.456418 7466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:06.456488 7466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:06.460589 7466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:06.460680 7466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:06.460718 7466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:06.460772 7466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:06.460810 7466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:06.466244 7466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45599
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:06.467620 7466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:06.468844 7466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:39:06.479477 7466 server_base.cc:1034] running on GCE node
W20260807 08:39:06.479372 7474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:06.479386 7473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:06.479378 7477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:06.480417 7466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:06.481882 7466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:06.483093 7466 hybrid_clock.cc:648] HybridClock initialized: now 1786091946483064 us; error 46 us; skew 500 ppm
Aug 07 08:39:06 dist-test-slave-h3ft krb5kdc[7449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091946, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:06.486508 7466 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:39:06.487676 7466 webserver.cc:460] Webserver started at http://127.0.0.1:37587/ using document root <none> and password file <none>
I20260807 08:39:06.488440 7466 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:06.488549 7466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:06.488835 7466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:06.491664 7466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "698c32c19af341a5b5149b6ab176afe2"
format_stamp: "Formatted at 2026-08-07 08:39:06 on dist-test-slave-h3ft"
server_key: "b2b6b277c9533c01bd17b915e5e3adad"
server_key_iv: "c69fc35579a00104b957e874bf9c90de"
server_key_version: "encryptionkey@0"
I20260807 08:39:06.492466 7466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "698c32c19af341a5b5149b6ab176afe2"
format_stamp: "Formatted at 2026-08-07 08:39:06 on dist-test-slave-h3ft"
server_key: "b2b6b277c9533c01bd17b915e5e3adad"
server_key_iv: "c69fc35579a00104b957e874bf9c90de"
server_key_version: "encryptionkey@0"
I20260807 08:39:06.497269 7466 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 08:39:06.500425 7485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:06.501948 7466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:39:06.502166 7466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "698c32c19af341a5b5149b6ab176afe2"
format_stamp: "Formatted at 2026-08-07 08:39:06 on dist-test-slave-h3ft"
server_key: "b2b6b277c9533c01bd17b915e5e3adad"
server_key_iv: "c69fc35579a00104b957e874bf9c90de"
server_key_version: "encryptionkey@0"
I20260807 08:39:06.502280 7466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:06.522920 7466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:06.526885 7466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:06.527208 7466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:06.536605 7466 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45599
I20260807 08:39:06.536641 7555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45599 every 8 connection(s)
I20260807 08:39:06.537837 7466 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:06.541635 7557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:06.542972 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 7466
I20260807 08:39:06.543179 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:06.543560 9912 external_mini_cluster.cc:1428] Setting key 989c985de379162b973d933fcfc98787
I20260807 08:39:06.548797 7557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2: Bootstrap starting.
I20260807 08:39:06.551618 7557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:06.552464 7557 log.cc:824] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:39:06 dist-test-slave-h3ft krb5kdc[7449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 08:39:06.554931 7557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2: No bootstrap required, opened a new log
I20260807 08:39:06.559065 7557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698c32c19af341a5b5149b6ab176afe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45599 } }
I20260807 08:39:06.559331 7557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:06.559422 7557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 698c32c19af341a5b5149b6ab176afe2, State: Initialized, Role: FOLLOWER
I20260807 08:39:06.559923 7560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:06.545288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51433 (local address 127.0.0.1:45599)
0807 08:39:06.545813 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:06.545830 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:06.545906 (+ 76us) server_negotiation.cc:407] Connection header received
0807 08:39:06.546627 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:06.546645 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:06.547121 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:06.547484 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:06.548430 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:06.549552 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:06.550508 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:06.551100 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:06.551279 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:06.554547 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:06.554553 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:06.554568 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:06.554597 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:06.556505 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:06.557082 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:06.557088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:06.557093 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:06.557212 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:06.557578 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:06.557581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:06.557582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:06.558019 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:06.558266 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:06.558615 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 08:39:06.558858 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":173,"threads_started":1}
I20260807 08:39:06.559919 7557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698c32c19af341a5b5149b6ab176afe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45599 } }
I20260807 08:39:06.560111 7557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:06.560209 7557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:06.560344 7557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:06.561568 7557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698c32c19af341a5b5149b6ab176afe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45599 } }
I20260807 08:39:06.561973 7557 leader_election.cc:302] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 698c32c19af341a5b5149b6ab176afe2; no voters:
I20260807 08:39:06.562371 7557 leader_election.cc:288] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:06.562471 7563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:06.562687 7563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [term 1 LEADER]: Becoming Leader. State: Replica: 698c32c19af341a5b5149b6ab176afe2, State: Running, Role: LEADER
I20260807 08:39:06.563117 7563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698c32c19af341a5b5149b6ab176afe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45599 } }
I20260807 08:39:06.563520 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:06.563943 7557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:06.565047 7564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "698c32c19af341a5b5149b6ab176afe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698c32c19af341a5b5149b6ab176afe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45599 } } }
I20260807 08:39:06.565215 7564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:06.565837 7565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 698c32c19af341a5b5149b6ab176afe2. Latest consensus state: current_term: 1 leader_uuid: "698c32c19af341a5b5149b6ab176afe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698c32c19af341a5b5149b6ab176afe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45599 } } }
I20260807 08:39:06.566641 7565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:06.567028 7582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:06.571416 7582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:06.577611 7560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:06.567537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37985 (local address 127.0.0.1:45599)
0807 08:39:06.567678 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:06.567685 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:06.567917 (+ 232us) server_negotiation.cc:407] Connection header received
0807 08:39:06.568087 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:06.568090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:06.568147 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:06.568229 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:06.568996 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:06.569561 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:06.570411 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:06.570668 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:06.570781 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:06.572474 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:06.572478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:06.572481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:06.572509 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:06.574623 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:06.575392 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:06.575396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:06.575398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:06.575500 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:06.575939 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:06.575943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:06.575945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:06.576194 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:06.576328 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:06.577350 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0807 08:39:06.577461 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:39:06.579857 7582 catalog_manager.cc:1347] Generated new cluster ID: 6ced6667e31e4b6896fb0d50ae4aef07
I20260807 08:39:06.579941 7582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:06.612952 7582 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:06.614568 7582 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:06.629359 7582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 698c32c19af341a5b5149b6ab176afe2: Generated new TSK 0
I20260807 08:39:06.630501 7582 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:06.642347 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 7466
W20260807 08:39:06.660722 7446 connection.cc:535] client connection to 127.0.0.1:45599 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45599): Connection reset by peer (error 104)
2026-08-07T08:39:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1783 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:06 dist-test-slave-h3ft krb5kdc[7599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:06 dist-test-slave-h3ft krb5kdc[7599](info): set up 2 sockets
Aug 07 08:39:06 dist-test-slave-h3ft krb5kdc[7599](info): commencing operation
krb5kdc: starting...
W20260807 08:39:08.012100 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.336s user 0.007s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:08 dist-test-slave-h3ft krb5kdc[7599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:39:08.162660 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40209
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:08.295877 7616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:08.296293 7616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:08.296375 7616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:08.300755 7616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:08.300844 7616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:08.300884 7616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:08.300940 7616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:08.300979 7616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:08.306739 7616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40209
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:08.308180 7616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:08.309505 7616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:08.319700 7626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:08.319707 7623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:08.319875 7616 server_base.cc:1034] running on GCE node
W20260807 08:39:08.319754 7624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:08.320833 7616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:08.322201 7616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:08.323380 7616 hybrid_clock.cc:648] HybridClock initialized: now 1786091948323342 us; error 48 us; skew 500 ppm
Aug 07 08:39:08 dist-test-slave-h3ft krb5kdc[7599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091948, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:08.326687 7616 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:39:08.327831 7616 webserver.cc:460] Webserver started at http://127.0.0.1:36371/ using document root <none> and password file <none>
I20260807 08:39:08.328486 7616 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:08.328552 7616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:08.328758 7616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:08.331395 7616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "1a7ad50003084843b8abc85277036d52"
format_stamp: "Formatted at 2026-08-07 08:39:08 on dist-test-slave-h3ft"
server_key: "c2a03b6c54540a356c4eb0a5dd66508f"
server_key_iv: "d45d4f4a2ab204f19eca49a34e917a6c"
server_key_version: "encryptionkey@0"
I20260807 08:39:08.332011 7616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "1a7ad50003084843b8abc85277036d52"
format_stamp: "Formatted at 2026-08-07 08:39:08 on dist-test-slave-h3ft"
server_key: "c2a03b6c54540a356c4eb0a5dd66508f"
server_key_iv: "d45d4f4a2ab204f19eca49a34e917a6c"
server_key_version: "encryptionkey@0"
I20260807 08:39:08.336555 7616 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:39:08.339666 7635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:08.341187 7616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:39:08.341346 7616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "1a7ad50003084843b8abc85277036d52"
format_stamp: "Formatted at 2026-08-07 08:39:08 on dist-test-slave-h3ft"
server_key: "c2a03b6c54540a356c4eb0a5dd66508f"
server_key_iv: "d45d4f4a2ab204f19eca49a34e917a6c"
server_key_version: "encryptionkey@0"
I20260807 08:39:08.341492 7616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:08.355783 7616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:08.359762 7616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:08.360018 7616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:08.369206 7616 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40209
I20260807 08:39:08.369231 7710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40209 every 8 connection(s)
I20260807 08:39:08.370460 7616 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:08.371946 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 7616
I20260807 08:39:08.372109 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:08.372603 9912 external_mini_cluster.cc:1428] Setting key e88a11467e7e201f46649a8ff74c7aa5
I20260807 08:39:08.374945 7711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:08.382863 7711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52: Bootstrap starting.
I20260807 08:39:08.383647 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:08.374459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34061 (local address 127.0.0.1:40209)
0807 08:39:08.375038 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:08.375068 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:08.375122 (+ 54us) server_negotiation.cc:407] Connection header received
0807 08:39:08.375940 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:08.375960 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:08.376451 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:08.376799 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:08.377203 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:08.378305 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:08.379611 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:08.380185 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:08.380326 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:08.381471 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:08.381477 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:08.381492 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:08.381971 (+ 479us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:08.382044 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:08.382071 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:08.382185 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:08.382472 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 08:39:08.382653 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":206,"threads_started":1}
I20260807 08:39:08.385908 7711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:08.386855 7711 log.cc:824] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:08.387331 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:08.389182 7711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52: No bootstrap required, opened a new log
I20260807 08:39:08.393417 7711 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7ad50003084843b8abc85277036d52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40209 } }
I20260807 08:39:08.393690 7711 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:08.393774 7711 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a7ad50003084843b8abc85277036d52, State: Initialized, Role: FOLLOWER
I20260807 08:39:08.394199 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:08.389871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38819 (local address 127.0.0.1:40209)
0807 08:39:08.389994 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:08.390002 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:08.390178 (+ 176us) server_negotiation.cc:407] Connection header received
0807 08:39:08.390338 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:08.390342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:08.390403 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:08.390489 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:08.390790 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:08.391446 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:08.392351 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:08.392593 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:08.392703 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:08.393681 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:08.393685 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:08.393689 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:08.393709 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:08.393745 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:08.393758 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:08.393853 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:08.393993 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 08:39:08.394038 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 08:39:08.394356 7711 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7ad50003084843b8abc85277036d52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40209 } }
I20260807 08:39:08.394555 7711 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:08.394649 7711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:08.394807 7711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:08.396052 7711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7ad50003084843b8abc85277036d52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40209 } }
I20260807 08:39:08.396464 7711 leader_election.cc:302] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a7ad50003084843b8abc85277036d52; no voters:
I20260807 08:39:08.396842 7711 leader_election.cc:288] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:08.396931 7725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:08.397172 7725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [term 1 LEADER]: Becoming Leader. State: Replica: 1a7ad50003084843b8abc85277036d52, State: Running, Role: LEADER
I20260807 08:39:08.397550 7725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7ad50003084843b8abc85277036d52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40209 } }
I20260807 08:39:08.398406 7711 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:08.398900 7725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a7ad50003084843b8abc85277036d52. Latest consensus state: current_term: 1 leader_uuid: "1a7ad50003084843b8abc85277036d52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7ad50003084843b8abc85277036d52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40209 } } }
I20260807 08:39:08.399102 7725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:08.399479 7728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a7ad50003084843b8abc85277036d52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7ad50003084843b8abc85277036d52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40209 } } }
I20260807 08:39:08.399606 7728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:08.401368 7735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:08.404631 7735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:08.411368 7735 catalog_manager.cc:1347] Generated new cluster ID: 7ce44a9ffb7c48e186c5b5646e7729cf
I20260807 08:39:08.411453 7735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:08.419912 7735 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:08.421415 7735 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:08.428520 7735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a7ad50003084843b8abc85277036d52: Generated new TSK 0
I20260807 08:39:08.429502 7735 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:08.438539 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 7616
W20260807 08:39:08.458657 7597 connection.cc:535] client connection to 127.0.0.1:40209 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40209): Connection reset by peer (error 104)
2026-08-07T08:39:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1798 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:08 dist-test-slave-h3ft krb5kdc[7760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:08 dist-test-slave-h3ft krb5kdc[7760](info): set up 2 sockets
Aug 07 08:39:08 dist-test-slave-h3ft krb5kdc[7760](info): commencing operation
krb5kdc: starting...
W20260807 08:39:09.775565 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:09 dist-test-slave-h3ft krb5kdc[7760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:39:09.942538 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39207
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:10.087852 7775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:10.088264 7775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:10.088346 7775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:10.092660 7775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:10.092789 7775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:10.092833 7775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:10.092888 7775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:10.092926 7775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:10.098937 7775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39207
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:10.100562 7775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:10.101985 7775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:10.112370 7785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:10.112370 7789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:10.112604 7786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:10.112643 7775 server_base.cc:1034] running on GCE node
I20260807 08:39:10.113556 7775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:10.114971 7775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:10.116210 7775 hybrid_clock.cc:648] HybridClock initialized: now 1786091950116182 us; error 47 us; skew 500 ppm
Aug 07 08:39:10 dist-test-slave-h3ft krb5kdc[7760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091950, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:10.123943 7775 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:39:10.125164 7775 webserver.cc:460] Webserver started at http://127.0.0.1:35559/ using document root <none> and password file <none>
I20260807 08:39:10.125867 7775 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:10.125963 7775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:10.126233 7775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:10.128794 7775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "cf874c421ad84921ba7a99535637518e"
format_stamp: "Formatted at 2026-08-07 08:39:10 on dist-test-slave-h3ft"
server_key: "25a1297dfdb164c0d8710357278126c7"
server_key_iv: "41f09ff5e02acc7592084690d21f4c5b"
server_key_version: "encryptionkey@0"
I20260807 08:39:10.129446 7775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "cf874c421ad84921ba7a99535637518e"
format_stamp: "Formatted at 2026-08-07 08:39:10 on dist-test-slave-h3ft"
server_key: "25a1297dfdb164c0d8710357278126c7"
server_key_iv: "41f09ff5e02acc7592084690d21f4c5b"
server_key_version: "encryptionkey@0"
I20260807 08:39:10.133975 7775 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 08:39:10.136907 7797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:10.138355 7775 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:39:10.138525 7775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "cf874c421ad84921ba7a99535637518e"
format_stamp: "Formatted at 2026-08-07 08:39:10 on dist-test-slave-h3ft"
server_key: "25a1297dfdb164c0d8710357278126c7"
server_key_iv: "41f09ff5e02acc7592084690d21f4c5b"
server_key_version: "encryptionkey@0"
I20260807 08:39:10.138656 7775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:10.144096 7775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:10.147801 7775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:10.148029 7775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:10.156734 7775 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39207
I20260807 08:39:10.156769 7865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39207 every 8 connection(s)
I20260807 08:39:10.157842 7775 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:10.161204 7866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:10.162807 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 7775
I20260807 08:39:10.162990 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:10.163434 9912 external_mini_cluster.cc:1428] Setting key 0f8b0357d79b4eeaf25b297d0dab0ced
I20260807 08:39:10.168509 7866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e: Bootstrap starting.
I20260807 08:39:10.169330 7869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:10.165095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38659 (local address 127.0.0.1:39207)
0807 08:39:10.165597 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:10.165614 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:10.165656 (+ 42us) server_negotiation.cc:407] Connection header received
0807 08:39:10.166367 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:10.166398 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:10.166709 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:10.167096 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:10.167982 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:10.167988 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:10.168000 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:10.168011 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:10.168072 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:10.168094 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:10.168257 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:10.168328 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 08:39:10.168506 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":150,"threads_started":1}
I20260807 08:39:10.171440 7866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:10.172406 7866 log.cc:824] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:10.172504 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:10.174482 7866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e: No bootstrap required, opened a new log
I20260807 08:39:10.176780 7869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:10.175042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54053 (local address 127.0.0.1:39207)
0807 08:39:10.175169 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:10.175173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:10.175400 (+ 227us) server_negotiation.cc:407] Connection header received
0807 08:39:10.175519 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:10.175521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:10.175550 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:10.175615 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:10.176334 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:10.176337 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:10.176339 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:10.176348 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:10.176385 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:10.176395 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:10.176470 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:10.176583 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 08:39:10.176616 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 08:39:10.179301 7866 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf874c421ad84921ba7a99535637518e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39207 } }
I20260807 08:39:10.179553 7866 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:10.179636 7866 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf874c421ad84921ba7a99535637518e, State: Initialized, Role: FOLLOWER
I20260807 08:39:10.180150 7866 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf874c421ad84921ba7a99535637518e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39207 } }
I20260807 08:39:10.180307 7866 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:10.180388 7866 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:10.180512 7866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:10.181530 7866 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf874c421ad84921ba7a99535637518e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39207 } }
I20260807 08:39:10.181922 7866 leader_election.cc:302] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf874c421ad84921ba7a99535637518e; no voters:
I20260807 08:39:10.182260 7866 leader_election.cc:288] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:10.182367 7881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:10.182624 7881 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [term 1 LEADER]: Becoming Leader. State: Replica: cf874c421ad84921ba7a99535637518e, State: Running, Role: LEADER
I20260807 08:39:10.183055 7881 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf874c421ad84921ba7a99535637518e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39207 } }
I20260807 08:39:10.183895 7866 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:10.185214 7883 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf874c421ad84921ba7a99535637518e. Latest consensus state: current_term: 1 leader_uuid: "cf874c421ad84921ba7a99535637518e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf874c421ad84921ba7a99535637518e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39207 } } }
I20260807 08:39:10.185233 7882 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf874c421ad84921ba7a99535637518e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf874c421ad84921ba7a99535637518e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39207 } } }
I20260807 08:39:10.185376 7883 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:10.185377 7882 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:10.185851 7893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:10.189627 7893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:10.196969 7893 catalog_manager.cc:1347] Generated new cluster ID: bd3b03af222c4e6ba5f5ab472d4dea03
I20260807 08:39:10.197109 7893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:10.205134 7893 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:10.205564 7893 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:10.215376 7893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf874c421ad84921ba7a99535637518e: Generated new TSK 0
I20260807 08:39:10.216398 7893 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:10.225813 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 7775
2026-08-07T08:39:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1778 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:10 dist-test-slave-h3ft krb5kdc[7909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:10 dist-test-slave-h3ft krb5kdc[7909](info): set up 2 sockets
Aug 07 08:39:10 dist-test-slave-h3ft krb5kdc[7909](info): commencing operation
krb5kdc: starting...
W20260807 08:39:11.552178 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:11 dist-test-slave-h3ft krb5kdc[7909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:39:11.708753 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35815
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:11.851269 7927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:11.851643 7927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:11.851696 7927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:11.856711 7927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:11.856812 7927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:11.856842 7927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:11.856864 7927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:11.856884 7927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:11.862883 7927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:11.864298 7927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:11.865571 7927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:11.876667 7933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:11.876803 7935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:11.876981 7927 server_base.cc:1034] running on GCE node
W20260807 08:39:11.876686 7932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:11.878055 7927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:11.879871 7927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:11.881107 7927 hybrid_clock.cc:648] HybridClock initialized: now 1786091951881073 us; error 52 us; skew 500 ppm
Aug 07 08:39:11 dist-test-slave-h3ft krb5kdc[7909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091951, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:11.885046 7927 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:39:11.886303 7927 webserver.cc:460] Webserver started at http://127.0.0.1:36771/ using document root <none> and password file <none>
I20260807 08:39:11.887140 7927 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:11.887245 7927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:11.887543 7927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:11.890292 7927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "87a09412ae114e0c97699e0382962bfe"
format_stamp: "Formatted at 2026-08-07 08:39:11 on dist-test-slave-h3ft"
server_key: "a64a0c364ee69b1bbc18187ce8c6a44b"
server_key_iv: "e3ebd63896167cf13c36fdd44eda6152"
server_key_version: "encryptionkey@0"
I20260807 08:39:11.891103 7927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "87a09412ae114e0c97699e0382962bfe"
format_stamp: "Formatted at 2026-08-07 08:39:11 on dist-test-slave-h3ft"
server_key: "a64a0c364ee69b1bbc18187ce8c6a44b"
server_key_iv: "e3ebd63896167cf13c36fdd44eda6152"
server_key_version: "encryptionkey@0"
I20260807 08:39:11.896060 7927 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 08:39:11.899689 7944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:11.901463 7927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:39:11.901674 7927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "87a09412ae114e0c97699e0382962bfe"
format_stamp: "Formatted at 2026-08-07 08:39:11 on dist-test-slave-h3ft"
server_key: "a64a0c364ee69b1bbc18187ce8c6a44b"
server_key_iv: "e3ebd63896167cf13c36fdd44eda6152"
server_key_version: "encryptionkey@0"
I20260807 08:39:11.901890 7927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:11.920634 7927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:11.924752 7927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:11.925081 7927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:11.934640 7927 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35815
I20260807 08:39:11.934676 8020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35815 every 8 connection(s)
I20260807 08:39:11.936012 7927 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:11.939272 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 7927
I20260807 08:39:11.939471 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:11.939854 8022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:11.939863 9912 external_mini_cluster.cc:1428] Setting key 8c60261c64ccb13196323256c2ec8e61
I20260807 08:39:11.949638 8022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe: Bootstrap starting.
Aug 07 08:39:11 dist-test-slave-h3ft krb5kdc[7909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 08:39:11.952809 8022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:11.954116 8022 log.cc:824] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:11.957372 8022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe: No bootstrap required, opened a new log
I20260807 08:39:11.959179 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:11.941756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43965 (local address 127.0.0.1:35815)
0807 08:39:11.942325 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:11.942365 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:11.942422 (+ 57us) server_negotiation.cc:407] Connection header received
0807 08:39:11.943726 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:11.943762 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:11.944319 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:11.944799 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:11.945701 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:11.947482 (+ 1781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:11.948599 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:11.949392 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:11.949575 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:11.952813 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:11.952824 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:11.952849 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:11.952890 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:11.955250 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:11.955789 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:11.955797 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:11.955804 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:11.955973 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:11.956299 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:11.956302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:11.956304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:11.956950 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:11.957189 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:11.957509 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 08:39:11.957912 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":150,"threads_started":1}
I20260807 08:39:11.963073 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:11.963150 8022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a09412ae114e0c97699e0382962bfe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35815 } }
I20260807 08:39:11.963500 8022 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:11.963567 8022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a09412ae114e0c97699e0382962bfe, State: Initialized, Role: FOLLOWER
I20260807 08:39:11.964135 8022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a09412ae114e0c97699e0382962bfe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35815 } }
I20260807 08:39:11.964329 8022 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:11.964435 8022 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:11.964564 8022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:11.965895 8022 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a09412ae114e0c97699e0382962bfe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35815 } }
I20260807 08:39:11.966327 8022 leader_election.cc:302] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87a09412ae114e0c97699e0382962bfe; no voters:
I20260807 08:39:11.966787 8022 leader_election.cc:288] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:11.966873 8039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:11.967187 8039 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [term 1 LEADER]: Becoming Leader. State: Replica: 87a09412ae114e0c97699e0382962bfe, State: Running, Role: LEADER
I20260807 08:39:11.967568 8039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a09412ae114e0c97699e0382962bfe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35815 } }
I20260807 08:39:11.968578 8022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:11.969038 8040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87a09412ae114e0c97699e0382962bfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a09412ae114e0c97699e0382962bfe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35815 } } }
I20260807 08:39:11.969182 8040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:11.969450 8040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87a09412ae114e0c97699e0382962bfe. Latest consensus state: current_term: 1 leader_uuid: "87a09412ae114e0c97699e0382962bfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a09412ae114e0c97699e0382962bfe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35815 } } }
I20260807 08:39:11.969547 8040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:11.970043 8044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:11.975247 8044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:11.975476 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:11.965833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40593 (local address 127.0.0.1:35815)
0807 08:39:11.965988 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:11.965993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:11.966077 (+ 84us) server_negotiation.cc:407] Connection header received
0807 08:39:11.966243 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:11.966247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:11.966306 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:11.966394 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:11.967234 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:11.967915 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:11.969047 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:11.969459 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:11.969545 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:11.971615 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:11.971619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:11.971624 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:11.971654 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:11.973442 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:11.973953 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:11.973957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:11.973959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:11.974066 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:11.974432 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:11.974435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:11.974437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:11.974678 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:11.974824 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:11.975164 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 08:39:11.975296 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":64}
I20260807 08:39:11.982729 8044 catalog_manager.cc:1347] Generated new cluster ID: 0dbf00cc487d4ec5b3d53926ddb6f82f
I20260807 08:39:11.982833 8044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:12.008989 8044 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:12.010618 8044 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:12.027863 8044 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87a09412ae114e0c97699e0382962bfe: Generated new TSK 0
I20260807 08:39:12.029115 8044 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:12.036231 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 7927
2026-08-07T08:39:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1814 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:12 dist-test-slave-h3ft krb5kdc[8071](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:12 dist-test-slave-h3ft krb5kdc[8071](info): set up 2 sockets
Aug 07 08:39:12 dist-test-slave-h3ft krb5kdc[8071](info): commencing operation
krb5kdc: starting...
W20260807 08:39:13.352316 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:13 dist-test-slave-h3ft krb5kdc[8071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:39:13.499468 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33565
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:13.635521 8087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:13.635874 8087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:13.635921 8087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:13.640229 8087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:13.640307 8087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:13.640327 8087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:13.640344 8087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:13.640360 8087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:13.646133 8087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:13.647557 8087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:13.648864 8087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:13.659137 8093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:13.659116 8097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:13.659147 8095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:13.659369 8087 server_base.cc:1034] running on GCE node
I20260807 08:39:13.660137 8087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:13.661652 8087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:13.662897 8087 hybrid_clock.cc:648] HybridClock initialized: now 1786091953662864 us; error 43 us; skew 500 ppm
Aug 07 08:39:13 dist-test-slave-h3ft krb5kdc[8071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:13.666707 8087 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:39:13.668118 8087 webserver.cc:460] Webserver started at http://127.0.0.1:38333/ using document root <none> and password file <none>
I20260807 08:39:13.668900 8087 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:13.669018 8087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:13.669353 8087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:13.672387 8087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "56d23c123005472db7fe0c36588d5431"
format_stamp: "Formatted at 2026-08-07 08:39:13 on dist-test-slave-h3ft"
server_key: "54681826afbd423ee2594c98717acdae"
server_key_iv: "8df31d0f6ab3075465c2a8346d9e42d7"
server_key_version: "encryptionkey@0"
I20260807 08:39:13.673256 8087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "56d23c123005472db7fe0c36588d5431"
format_stamp: "Formatted at 2026-08-07 08:39:13 on dist-test-slave-h3ft"
server_key: "54681826afbd423ee2594c98717acdae"
server_key_iv: "8df31d0f6ab3075465c2a8346d9e42d7"
server_key_version: "encryptionkey@0"
I20260807 08:39:13.678490 8087 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:39:13.681656 8107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:13.683308 8087 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:39:13.683482 8087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "56d23c123005472db7fe0c36588d5431"
format_stamp: "Formatted at 2026-08-07 08:39:13 on dist-test-slave-h3ft"
server_key: "54681826afbd423ee2594c98717acdae"
server_key_iv: "8df31d0f6ab3075465c2a8346d9e42d7"
server_key_version: "encryptionkey@0"
I20260807 08:39:13.683650 8087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:13.727944 8087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:13.732254 8087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:13.732534 8087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:13.742197 8087 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33565
I20260807 08:39:13.742240 8188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33565 every 8 connection(s)
I20260807 08:39:13.743603 8087 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:13.747339 8189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:13.749740 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 8087
I20260807 08:39:13.749922 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:13.750296 9912 external_mini_cluster.cc:1428] Setting key 7e42320c85976814c87366b25b50e784
I20260807 08:39:13.756390 8189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431: Bootstrap starting.
I20260807 08:39:13.759850 8189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:13.760926 8189 log.cc:824] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:13.762701 8192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:13.752158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43829 (local address 127.0.0.1:33565)
0807 08:39:13.752730 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:13.752749 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:13.752795 (+ 46us) server_negotiation.cc:407] Connection header received
0807 08:39:13.753822 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:13.753854 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:13.754534 (+ 680us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:13.755072 (+ 538us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:13.755625 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:13.757273 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:13.758353 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:13.758916 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:13.759153 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:13.760408 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:13.760415 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:13.760435 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:13.760999 (+ 564us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:13.761125 (+ 126us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:13.761157 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:13.761291 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:13.761420 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 08:39:13.761724 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":179,"threads_started":1}
I20260807 08:39:13.763700 8189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431: No bootstrap required, opened a new log
I20260807 08:39:13.766706 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:13.769517 8189 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d23c123005472db7fe0c36588d5431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33565 } }
I20260807 08:39:13.770081 8189 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:13.770185 8189 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56d23c123005472db7fe0c36588d5431, State: Initialized, Role: FOLLOWER
I20260807 08:39:13.770844 8189 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d23c123005472db7fe0c36588d5431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33565 } }
I20260807 08:39:13.771097 8189 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:13.771210 8189 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:13.771340 8189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:13.772921 8189 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d23c123005472db7fe0c36588d5431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33565 } }
I20260807 08:39:13.773443 8189 leader_election.cc:302] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56d23c123005472db7fe0c36588d5431; no voters:
I20260807 08:39:13.773916 8189 leader_election.cc:288] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:13.774060 8200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:13.774380 8200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [term 1 LEADER]: Becoming Leader. State: Replica: 56d23c123005472db7fe0c36588d5431, State: Running, Role: LEADER
I20260807 08:39:13.774842 8192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:13.769809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38765 (local address 127.0.0.1:33565)
0807 08:39:13.769985 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:13.769990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:13.770221 (+ 231us) server_negotiation.cc:407] Connection header received
0807 08:39:13.770383 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:13.770386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:13.770442 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:13.770554 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:13.770890 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:13.771711 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:13.772917 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:13.773233 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:13.773317 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:13.774335 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:13.774339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:13.774342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:13.774358 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:13.774393 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:13.774407 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:13.774455 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:13.774623 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 08:39:13.774691 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 08:39:13.774827 8200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d23c123005472db7fe0c36588d5431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33565 } }
I20260807 08:39:13.775816 8189 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:13.777364 8202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56d23c123005472db7fe0c36588d5431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d23c123005472db7fe0c36588d5431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33565 } } }
I20260807 08:39:13.777359 8204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56d23c123005472db7fe0c36588d5431. Latest consensus state: current_term: 1 leader_uuid: "56d23c123005472db7fe0c36588d5431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d23c123005472db7fe0c36588d5431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33565 } } }
I20260807 08:39:13.777518 8204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:13.777518 8202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:13.778010 8217 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:13.781775 8217 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:13.789125 8217 catalog_manager.cc:1347] Generated new cluster ID: 8f106197f89649df9455daa4f594a6cb
I20260807 08:39:13.789248 8217 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:13.802843 8217 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:13.804450 8217 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:13.815789 8217 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56d23c123005472db7fe0c36588d5431: Generated new TSK 0
I20260807 08:39:13.817111 8217 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:13.826771 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 8087
2026-08-07T08:39:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1790 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:13 dist-test-slave-h3ft krb5kdc[8231](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:13 dist-test-slave-h3ft krb5kdc[8231](info): set up 2 sockets
Aug 07 08:39:13 dist-test-slave-h3ft krb5kdc[8231](info): commencing operation
krb5kdc: starting...
W20260807 08:39:15.162012 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:15 dist-test-slave-h3ft krb5kdc[8231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:39:15.318994 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38609
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:15.453748 8250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:15.454137 8250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:15.454185 8250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:15.458487 8250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:15.458559 8250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:15.458578 8250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:15.458649 8250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:15.458684 8250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:15.464409 8250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:15.465792 8250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:15.467105 8250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:15.477290 8256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:15.477310 8258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:15.477367 8255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:15.477406 8250 server_base.cc:1034] running on GCE node
I20260807 08:39:15.478404 8250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:15.479846 8250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:15.481019 8250 hybrid_clock.cc:648] HybridClock initialized: now 1786091955480999 us; error 33 us; skew 500 ppm
Aug 07 08:39:15 dist-test-slave-h3ft krb5kdc[8231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091955, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:15.490372 8250 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:39:15.491705 8250 webserver.cc:460] Webserver started at http://127.0.0.1:36707/ using document root <none> and password file <none>
I20260807 08:39:15.492442 8250 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:15.492514 8250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:15.492820 8250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:15.495505 8250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "8e9adfb3d75046ea8cc2c573207024f8"
format_stamp: "Formatted at 2026-08-07 08:39:15 on dist-test-slave-h3ft"
server_key: "d3363ac7fd1a990859166329f8ffc68e"
server_key_iv: "e2b2fa4cff47393baec2a62f993ca1eb"
server_key_version: "encryptionkey@0"
I20260807 08:39:15.496162 8250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "8e9adfb3d75046ea8cc2c573207024f8"
format_stamp: "Formatted at 2026-08-07 08:39:15 on dist-test-slave-h3ft"
server_key: "d3363ac7fd1a990859166329f8ffc68e"
server_key_iv: "e2b2fa4cff47393baec2a62f993ca1eb"
server_key_version: "encryptionkey@0"
I20260807 08:39:15.500834 8250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:39:15.504088 8267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:15.505640 8250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:39:15.505810 8250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "8e9adfb3d75046ea8cc2c573207024f8"
format_stamp: "Formatted at 2026-08-07 08:39:15 on dist-test-slave-h3ft"
server_key: "d3363ac7fd1a990859166329f8ffc68e"
server_key_iv: "e2b2fa4cff47393baec2a62f993ca1eb"
server_key_version: "encryptionkey@0"
I20260807 08:39:15.505956 8250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:15.511550 8250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:15.515458 8250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:15.515689 8250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:15.524228 8250 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38609
I20260807 08:39:15.524286 8335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38609 every 8 connection(s)
I20260807 08:39:15.525378 8250 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:15.528720 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 8250
I20260807 08:39:15.528789 8336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:15.528882 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:15.529286 9912 external_mini_cluster.cc:1428] Setting key f91c10edd730b322733c4903d2d5eca4
I20260807 08:39:15.535216 8339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:15.530867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37689 (local address 127.0.0.1:38609)
0807 08:39:15.531408 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:15.531425 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:15.531466 (+ 41us) server_negotiation.cc:407] Connection header received
0807 08:39:15.532144 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:15.532163 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:15.532474 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:15.532815 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:15.533751 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:15.533756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:15.533768 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:15.533779 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:15.533840 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:15.533861 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:15.533982 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:15.534123 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 08:39:15.534319 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":223,"threads_started":1}
I20260807 08:39:15.536845 8336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8: Bootstrap starting.
I20260807 08:39:15.538271 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:15.539856 8336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:15.540972 8336 log.cc:824] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:15.542984 8339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:15.541052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50669 (local address 127.0.0.1:38609)
0807 08:39:15.541194 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:15.541197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:15.541532 (+ 335us) server_negotiation.cc:407] Connection header received
0807 08:39:15.541629 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:15.541635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:15.541678 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:15.541771 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:15.542525 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:15.542528 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:15.542530 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:15.542540 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:15.542577 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:15.542588 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:15.542657 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:15.542778 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 08:39:15.542815 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 08:39:15.543628 8336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8: No bootstrap required, opened a new log
I20260807 08:39:15.548233 8336 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9adfb3d75046ea8cc2c573207024f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38609 } }
I20260807 08:39:15.548609 8336 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:15.548684 8336 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9adfb3d75046ea8cc2c573207024f8, State: Initialized, Role: FOLLOWER
I20260807 08:39:15.549237 8336 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9adfb3d75046ea8cc2c573207024f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38609 } }
I20260807 08:39:15.549401 8336 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:15.549482 8336 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:15.549614 8336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:15.550693 8336 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9adfb3d75046ea8cc2c573207024f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38609 } }
I20260807 08:39:15.551157 8336 leader_election.cc:302] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e9adfb3d75046ea8cc2c573207024f8; no voters:
I20260807 08:39:15.551565 8336 leader_election.cc:288] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:15.551707 8348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:15.552034 8348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9adfb3d75046ea8cc2c573207024f8, State: Running, Role: LEADER
I20260807 08:39:15.552393 8348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9adfb3d75046ea8cc2c573207024f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38609 } }
I20260807 08:39:15.553064 8336 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:15.554378 8350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e9adfb3d75046ea8cc2c573207024f8. Latest consensus state: current_term: 1 leader_uuid: "8e9adfb3d75046ea8cc2c573207024f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9adfb3d75046ea8cc2c573207024f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38609 } } }
I20260807 08:39:15.554535 8350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:15.554927 8349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e9adfb3d75046ea8cc2c573207024f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9adfb3d75046ea8cc2c573207024f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38609 } } }
I20260807 08:39:15.555159 8349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:15.555213 8362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:15.559059 8362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:15.565773 8362 catalog_manager.cc:1347] Generated new cluster ID: 5e0b55d5146540fda4210a923d9fc546
I20260807 08:39:15.565865 8362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:15.577126 8362 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:15.577522 8362 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:15.585815 8362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e9adfb3d75046ea8cc2c573207024f8: Generated new TSK 0
I20260807 08:39:15.586755 8362 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:15.593554 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 8250
2026-08-07T08:39:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1766 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:15 dist-test-slave-h3ft krb5kdc[8378](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:15 dist-test-slave-h3ft krb5kdc[8378](info): set up 2 sockets
Aug 07 08:39:15 dist-test-slave-h3ft krb5kdc[8378](info): commencing operation
krb5kdc: starting...
W20260807 08:39:16.870236 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:16 dist-test-slave-h3ft krb5kdc[8378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:39:17.019629 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43281
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:17.154222 8397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:17.154575 8397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:17.154623 8397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:17.158887 8397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:17.158968 8397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:17.158989 8397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:17.159075 8397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:17.159111 8397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:17.164989 8397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:17.166456 8397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:17.167814 8397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:17.178149 8406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:17.178190 8409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:17.178215 8405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:17.178597 8397 server_base.cc:1034] running on GCE node
I20260807 08:39:17.179387 8397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:17.180773 8397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:17.181968 8397 hybrid_clock.cc:648] HybridClock initialized: now 1786091957181948 us; error 58 us; skew 500 ppm
Aug 07 08:39:17 dist-test-slave-h3ft krb5kdc[8378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091957, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:17.185281 8397 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:39:17.186378 8397 webserver.cc:460] Webserver started at http://127.0.0.1:40181/ using document root <none> and password file <none>
I20260807 08:39:17.187111 8397 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:17.187188 8397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:17.187405 8397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:17.189973 8397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "6db22a966ce847e6bc3b9267bea751a9"
format_stamp: "Formatted at 2026-08-07 08:39:17 on dist-test-slave-h3ft"
server_key: "ad9ce6c9e3ee81396eb80ac7786e1bc6"
server_key_iv: "2162b09d0f9ea9e8e0e33fb58c815980"
server_key_version: "encryptionkey@0"
I20260807 08:39:17.190598 8397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "6db22a966ce847e6bc3b9267bea751a9"
format_stamp: "Formatted at 2026-08-07 08:39:17 on dist-test-slave-h3ft"
server_key: "ad9ce6c9e3ee81396eb80ac7786e1bc6"
server_key_iv: "2162b09d0f9ea9e8e0e33fb58c815980"
server_key_version: "encryptionkey@0"
I20260807 08:39:17.195406 8397 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 08:39:17.198414 8420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:17.200002 8397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:39:17.200125 8397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "6db22a966ce847e6bc3b9267bea751a9"
format_stamp: "Formatted at 2026-08-07 08:39:17 on dist-test-slave-h3ft"
server_key: "ad9ce6c9e3ee81396eb80ac7786e1bc6"
server_key_iv: "2162b09d0f9ea9e8e0e33fb58c815980"
server_key_version: "encryptionkey@0"
I20260807 08:39:17.200240 8397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:17.219460 8397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:17.223196 8397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:17.223408 8397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:17.232419 8397 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43281
I20260807 08:39:17.232450 8490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43281 every 8 connection(s)
I20260807 08:39:17.233736 8397 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:17.237372 8491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:17.239508 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 8397
I20260807 08:39:17.239652 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:17.240032 9912 external_mini_cluster.cc:1428] Setting key 87b6cce3c9c4ab13449220ed524431ec
I20260807 08:39:17.246093 8491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9: Bootstrap starting.
I20260807 08:39:17.249553 8491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:17.250523 8491 log.cc:824] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:39:17 dist-test-slave-h3ft krb5kdc[8378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 08:39:17.253374 8491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9: No bootstrap required, opened a new log
I20260807 08:39:17.257887 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:17.241988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:43369 (local address 127.0.0.1:43281)
0807 08:39:17.242570 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:17.242594 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:17.242647 (+ 53us) server_negotiation.cc:407] Connection header received
0807 08:39:17.243734 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:17.243767 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:17.244312 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:17.244774 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:17.245608 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:17.247204 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:17.248123 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:17.248868 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:17.249034 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:17.252283 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:17.252289 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:17.252308 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:17.252341 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:17.254344 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:17.254971 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:17.254977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:17.254983 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:17.255182 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:17.255543 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:17.255546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:17.255547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:17.256017 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:17.256217 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:17.256630 (+ 413us) server_negotiation.cc:299] Negotiation successful
0807 08:39:17.256930 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":163,"threads_started":1}
I20260807 08:39:17.258391 8491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db22a966ce847e6bc3b9267bea751a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43281 } }
I20260807 08:39:17.258725 8491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:17.258788 8491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6db22a966ce847e6bc3b9267bea751a9, State: Initialized, Role: FOLLOWER
I20260807 08:39:17.259325 8491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db22a966ce847e6bc3b9267bea751a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43281 } }
I20260807 08:39:17.259522 8491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:17.259606 8491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:17.259739 8491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:17.261111 8491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db22a966ce847e6bc3b9267bea751a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43281 } }
I20260807 08:39:17.261567 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:17.261559 8491 leader_election.cc:302] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6db22a966ce847e6bc3b9267bea751a9; no voters:
I20260807 08:39:17.261960 8491 leader_election.cc:288] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:17.262097 8496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:17.262346 8496 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [term 1 LEADER]: Becoming Leader. State: Replica: 6db22a966ce847e6bc3b9267bea751a9, State: Running, Role: LEADER
I20260807 08:39:17.262799 8496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db22a966ce847e6bc3b9267bea751a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43281 } }
I20260807 08:39:17.263567 8491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:17.264827 8497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6db22a966ce847e6bc3b9267bea751a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db22a966ce847e6bc3b9267bea751a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43281 } } }
I20260807 08:39:17.264986 8497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:17.265257 8497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6db22a966ce847e6bc3b9267bea751a9. Latest consensus state: current_term: 1 leader_uuid: "6db22a966ce847e6bc3b9267bea751a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db22a966ce847e6bc3b9267bea751a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43281 } } }
I20260807 08:39:17.265359 8497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:17.266757 8521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:17.270879 8521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:17.275571 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:17.264484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:41845 (local address 127.0.0.1:43281)
0807 08:39:17.264638 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:17.264644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:17.265361 (+ 717us) server_negotiation.cc:407] Connection header received
0807 08:39:17.265579 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:17.265582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:17.265641 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:17.265703 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:17.266751 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:17.267444 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:17.268609 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:17.268942 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:17.269082 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:17.270994 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:17.270998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:17.271029 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:17.271058 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:17.272991 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:17.273800 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:17.273807 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:17.273811 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:17.273920 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:17.274344 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:17.274349 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:17.274353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:17.274626 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:17.274798 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:17.275211 (+ 413us) server_negotiation.cc:299] Negotiation successful
0807 08:39:17.275353 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:39:17.278507 8521 catalog_manager.cc:1347] Generated new cluster ID: 48b174725dc343bf9f88b4aad6f759b3
I20260807 08:39:17.278652 8521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:17.291060 8521 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:17.292832 8521 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:17.302210 8521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6db22a966ce847e6bc3b9267bea751a9: Generated new TSK 0
I20260807 08:39:17.303143 8521 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:17.311724 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 8397
W20260807 08:39:17.326349 8376 connection.cc:535] client connection to 127.0.0.1:43281 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43281): Connection reset by peer (error 104)
2026-08-07T08:39:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1717 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:17 dist-test-slave-h3ft krb5kdc[8535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:17 dist-test-slave-h3ft krb5kdc[8535](info): set up 2 sockets
Aug 07 08:39:17 dist-test-slave-h3ft krb5kdc[8535](info): commencing operation
krb5kdc: starting...
W20260807 08:39:18.585326 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:18 dist-test-slave-h3ft krb5kdc[8535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:39:18.744843 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34211
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:18.887262 8554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:18.887632 8554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:18.887683 8554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:18.892143 8554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:18.892241 8554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:18.892261 8554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:18.892278 8554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:18.892294 8554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:18.898437 8554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:18.900059 8554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:18.901424 8554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:18.912194 8560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:18.912385 8554 server_base.cc:1034] running on GCE node
W20260807 08:39:18.912230 8565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:18.912213 8562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:18.913419 8554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:18.914891 8554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:18.916095 8554 hybrid_clock.cc:648] HybridClock initialized: now 1786091958916081 us; error 31 us; skew 500 ppm
Aug 07 08:39:18 dist-test-slave-h3ft krb5kdc[8535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091958, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:18.919536 8554 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:39:18.920686 8554 webserver.cc:460] Webserver started at http://127.0.0.1:38417/ using document root <none> and password file <none>
I20260807 08:39:18.921356 8554 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:18.921422 8554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:18.921628 8554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:18.924320 8554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "aa7d0eebde524dc29d530cf4dc84ab2a"
format_stamp: "Formatted at 2026-08-07 08:39:18 on dist-test-slave-h3ft"
server_key: "880cc634f2332fe79039442d49a3cdbe"
server_key_iv: "81b1f83d7de11a62472b5b6520892550"
server_key_version: "encryptionkey@0"
I20260807 08:39:18.924959 8554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "aa7d0eebde524dc29d530cf4dc84ab2a"
format_stamp: "Formatted at 2026-08-07 08:39:18 on dist-test-slave-h3ft"
server_key: "880cc634f2332fe79039442d49a3cdbe"
server_key_iv: "81b1f83d7de11a62472b5b6520892550"
server_key_version: "encryptionkey@0"
I20260807 08:39:18.929615 8554 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 08:39:18.932822 8577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:18.934432 8554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 08:39:18.934621 8554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "aa7d0eebde524dc29d530cf4dc84ab2a"
format_stamp: "Formatted at 2026-08-07 08:39:18 on dist-test-slave-h3ft"
server_key: "880cc634f2332fe79039442d49a3cdbe"
server_key_iv: "81b1f83d7de11a62472b5b6520892550"
server_key_version: "encryptionkey@0"
I20260807 08:39:18.934780 8554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:18.954741 8554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:18.958683 8554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:18.958959 8554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:18.968333 8554 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34211
I20260807 08:39:18.968370 8645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34211 every 8 connection(s)
I20260807 08:39:18.969652 8554 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:18.973408 8646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:18.975759 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 8554
I20260807 08:39:18.975965 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:18.976387 9912 external_mini_cluster.cc:1428] Setting key a226ec1ed81905cdba136e076389e794
I20260807 08:39:18.981956 8646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a: Bootstrap starting.
I20260807 08:39:18.985242 8646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:18.986172 8646 log.cc:824] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:18.987771 8650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:18.978529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:50699 (local address 127.0.0.1:34211)
0807 08:39:18.979157 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:18.979177 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:18.979223 (+ 46us) server_negotiation.cc:407] Connection header received
0807 08:39:18.979981 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:18.980001 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:18.980428 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:18.980833 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:18.981325 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:18.982519 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:18.983596 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:18.984130 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:18.984322 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:18.985436 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:18.985441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:18.985457 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:18.985938 (+ 481us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:18.986022 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:18.986049 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:18.986209 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:18.986413 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 08:39:18.986591 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":181,"threads_started":1}
I20260807 08:39:18.988932 8646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a: No bootstrap required, opened a new log
I20260807 08:39:18.991547 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:18.994050 8646 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7d0eebde524dc29d530cf4dc84ab2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34211 } }
I20260807 08:39:18.994473 8646 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:18.994549 8646 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa7d0eebde524dc29d530cf4dc84ab2a, State: Initialized, Role: FOLLOWER
I20260807 08:39:18.995193 8646 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7d0eebde524dc29d530cf4dc84ab2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34211 } }
I20260807 08:39:18.995369 8646 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:18.995458 8646 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:18.995576 8646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:18.996809 8646 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7d0eebde524dc29d530cf4dc84ab2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34211 } }
I20260807 08:39:18.997233 8646 leader_election.cc:302] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa7d0eebde524dc29d530cf4dc84ab2a; no voters:
I20260807 08:39:18.997673 8646 leader_election.cc:288] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:18.997758 8658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:18.998036 8658 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [term 1 LEADER]: Becoming Leader. State: Replica: aa7d0eebde524dc29d530cf4dc84ab2a, State: Running, Role: LEADER
I20260807 08:39:18.998442 8658 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7d0eebde524dc29d530cf4dc84ab2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34211 } }
I20260807 08:39:18.999053 8650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:18.994103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:58689 (local address 127.0.0.1:34211)
0807 08:39:18.994250 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:18.994256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:18.994442 (+ 186us) server_negotiation.cc:407] Connection header received
0807 08:39:18.994650 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:18.994653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:18.994703 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:18.994795 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:18.995183 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:18.995930 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:18.996799 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:18.997139 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:18.997260 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:18.998321 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:18.998325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:18.998328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:18.998343 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:18.998381 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:18.998394 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:18.998477 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:18.998842 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 08:39:18.998880 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:39:18.999435 8646 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:19.001616 8659 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa7d0eebde524dc29d530cf4dc84ab2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7d0eebde524dc29d530cf4dc84ab2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34211 } } }
I20260807 08:39:19.001616 8661 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa7d0eebde524dc29d530cf4dc84ab2a. Latest consensus state: current_term: 1 leader_uuid: "aa7d0eebde524dc29d530cf4dc84ab2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7d0eebde524dc29d530cf4dc84ab2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34211 } } }
I20260807 08:39:19.001796 8661 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:19.001796 8659 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a [sys.catalog]: This master's current role is: LEADER
W20260807 08:39:19.005720 8680 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:39:19.005803 8680 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:39:19.005887 8681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:19.006893 8681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:19.014240 8681 catalog_manager.cc:1347] Generated new cluster ID: b8576d8143754b9bb6c89f196b86a0d3
I20260807 08:39:19.014353 8681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:19.035393 8681 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:19.037071 8681 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:19.047426 8681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa7d0eebde524dc29d530cf4dc84ab2a: Generated new TSK 0
I20260807 08:39:19.048652 8681 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:19.058432 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 8554
W20260807 08:39:19.074009 8533 connection.cc:535] client connection to 127.0.0.1:34211 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34211): Connection reset by peer (error 104)
2026-08-07T08:39:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1747 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:19 dist-test-slave-h3ft krb5kdc[8688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:19 dist-test-slave-h3ft krb5kdc[8688](info): set up 2 sockets
Aug 07 08:39:19 dist-test-slave-h3ft krb5kdc[8688](info): commencing operation
krb5kdc: starting...
W20260807 08:39:20.370040 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:20 dist-test-slave-h3ft krb5kdc[8688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:39:20.519227 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41277
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:20.654619 8707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:20.655122 8707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:20.655256 8707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:20.659715 8707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:20.659835 8707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:20.659874 8707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:20.659924 8707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:20.659962 8707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:20.666256 8707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:20.668228 8707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:20.669770 8707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:20.680850 8715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:20.680850 8714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:20.680850 8717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:20.681526 8707 server_base.cc:1034] running on GCE node
I20260807 08:39:20.682240 8707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:20.684142 8707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:20.685349 8707 hybrid_clock.cc:648] HybridClock initialized: now 1786091960685331 us; error 33 us; skew 500 ppm
Aug 07 08:39:20 dist-test-slave-h3ft krb5kdc[8688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091960, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:20.696138 8707 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:39:20.697559 8707 webserver.cc:460] Webserver started at http://127.0.0.1:43295/ using document root <none> and password file <none>
I20260807 08:39:20.698372 8707 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:20.698453 8707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:20.698767 8707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:20.701565 8707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "de309740af9b42c19583faa33b4b9428"
format_stamp: "Formatted at 2026-08-07 08:39:20 on dist-test-slave-h3ft"
server_key: "c041632861d40d7fe18f443ac4011997"
server_key_iv: "2da08ed404dfc8e2e111dc6abbbd93fc"
server_key_version: "encryptionkey@0"
I20260807 08:39:20.702301 8707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "de309740af9b42c19583faa33b4b9428"
format_stamp: "Formatted at 2026-08-07 08:39:20 on dist-test-slave-h3ft"
server_key: "c041632861d40d7fe18f443ac4011997"
server_key_iv: "2da08ed404dfc8e2e111dc6abbbd93fc"
server_key_version: "encryptionkey@0"
I20260807 08:39:20.707330 8707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 08:39:20.710594 8726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:20.712306 8707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:39:20.712466 8707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "de309740af9b42c19583faa33b4b9428"
format_stamp: "Formatted at 2026-08-07 08:39:20 on dist-test-slave-h3ft"
server_key: "c041632861d40d7fe18f443ac4011997"
server_key_iv: "2da08ed404dfc8e2e111dc6abbbd93fc"
server_key_version: "encryptionkey@0"
I20260807 08:39:20.712618 8707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:20.718106 8707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:20.722116 8707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:20.722360 8707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:20.730801 8707 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41277
I20260807 08:39:20.730836 8804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41277 every 8 connection(s)
I20260807 08:39:20.732061 8707 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:20.735685 8805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:20.738983 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 8707
I20260807 08:39:20.739238 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:20.739606 9912 external_mini_cluster.cc:1428] Setting key ea6b49024bfe2755cba56e10ee2b33bd
I20260807 08:39:20.743069 8805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428: Bootstrap starting.
I20260807 08:39:20.746191 8810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:20.741365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:43413 (local address 127.0.0.1:41277)
0807 08:39:20.741896 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:20.741920 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:20.741972 (+ 52us) server_negotiation.cc:407] Connection header received
0807 08:39:20.742865 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:20.742890 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:20.743319 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:20.743731 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:20.744607 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:20.744614 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:20.744630 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:20.744643 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:20.744722 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:20.744755 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:20.744906 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:20.744979 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 08:39:20.745153 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":158,"threads_started":1}
I20260807 08:39:20.746392 8805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:20.747453 8805 log.cc:824] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:20.749668 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:20.750036 8805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428: No bootstrap required, opened a new log
I20260807 08:39:20.754151 8810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:20.752216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:41771 (local address 127.0.0.1:41277)
0807 08:39:20.752376 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:20.752380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:20.752633 (+ 253us) server_negotiation.cc:407] Connection header received
0807 08:39:20.752764 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:20.752766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:20.752798 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:20.752867 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:20.753686 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:20.753690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:20.753692 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:20.753703 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:20.753733 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:20.753744 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:20.753814 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:20.753933 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 08:39:20.753973 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 08:39:20.755019 8805 raft_consensus.cc:348] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de309740af9b42c19583faa33b4b9428" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41277 } }
I20260807 08:39:20.755352 8805 raft_consensus.cc:374] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:20.755421 8805 raft_consensus.cc:729] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de309740af9b42c19583faa33b4b9428, State: Initialized, Role: FOLLOWER
I20260807 08:39:20.755949 8805 consensus_queue.cc:260] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de309740af9b42c19583faa33b4b9428" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41277 } }
I20260807 08:39:20.756111 8805 raft_consensus.cc:388] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:20.756184 8805 raft_consensus.cc:482] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:20.756290 8805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:20.757369 8805 raft_consensus.cc:504] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de309740af9b42c19583faa33b4b9428" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41277 } }
I20260807 08:39:20.757764 8805 leader_election.cc:302] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de309740af9b42c19583faa33b4b9428; no voters:
I20260807 08:39:20.758118 8805 leader_election.cc:288] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:20.758288 8817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:20.758548 8817 raft_consensus.cc:686] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [term 1 LEADER]: Becoming Leader. State: Replica: de309740af9b42c19583faa33b4b9428, State: Running, Role: LEADER
I20260807 08:39:20.758930 8817 consensus_queue.cc:237] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de309740af9b42c19583faa33b4b9428" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41277 } }
I20260807 08:39:20.759619 8805 sys_catalog.cc:564] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:20.761252 8820 sys_catalog.cc:455] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de309740af9b42c19583faa33b4b9428. Latest consensus state: current_term: 1 leader_uuid: "de309740af9b42c19583faa33b4b9428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de309740af9b42c19583faa33b4b9428" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41277 } } }
I20260807 08:39:20.761412 8820 sys_catalog.cc:458] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:20.761600 8818 sys_catalog.cc:455] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de309740af9b42c19583faa33b4b9428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de309740af9b42c19583faa33b4b9428" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41277 } } }
I20260807 08:39:20.761709 8818 sys_catalog.cc:458] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:20.762421 8831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:20.765628 8831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:20.771652 8831 catalog_manager.cc:1347] Generated new cluster ID: 9c34d1210d4c4894b6aa887fd7703521
I20260807 08:39:20.771732 8831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:20.789741 8831 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:20.790678 8831 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:20.796267 8831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de309740af9b42c19583faa33b4b9428: Generated new TSK 0
I20260807 08:39:20.797377 8831 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:20.803800 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 8707
2026-08-07T08:39:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1748 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:20 dist-test-slave-h3ft krb5kdc[8845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:20 dist-test-slave-h3ft krb5kdc[8845](info): set up 2 sockets
Aug 07 08:39:20 dist-test-slave-h3ft krb5kdc[8845](info): commencing operation
krb5kdc: starting...
W20260807 08:39:22.095620 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.001s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:22 dist-test-slave-h3ft krb5kdc[8845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:39:22.242946 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46151
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:22.375664 8866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:22.376087 8866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:22.376173 8866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:22.380460 8866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:22.380563 8866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:22.380600 8866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:22.380664 8866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:22.380712 8866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:22.386622 8866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46151
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:22.388303 8866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:22.389626 8866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:22.399941 8873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:22.399967 8875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:22.399977 8877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:22.400166 8866 server_base.cc:1034] running on GCE node
I20260807 08:39:22.401021 8866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:22.402490 8866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:22.403658 8866 hybrid_clock.cc:648] HybridClock initialized: now 1786091962403616 us; error 60 us; skew 500 ppm
Aug 07 08:39:22 dist-test-slave-h3ft krb5kdc[8845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091962, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:22.406924 8866 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:39:22.408072 8866 webserver.cc:460] Webserver started at http://127.0.0.1:44385/ using document root <none> and password file <none>
I20260807 08:39:22.408787 8866 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:22.408892 8866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:22.409185 8866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:22.411998 8866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "6faf85ceaf204b4abbb9d3c102a5e1c6"
format_stamp: "Formatted at 2026-08-07 08:39:22 on dist-test-slave-h3ft"
server_key: "56a95595c9641bf9674c5f75dfa1394f"
server_key_iv: "7af6d64782e34e3a004dab5798acbadf"
server_key_version: "encryptionkey@0"
I20260807 08:39:22.412672 8866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "6faf85ceaf204b4abbb9d3c102a5e1c6"
format_stamp: "Formatted at 2026-08-07 08:39:22 on dist-test-slave-h3ft"
server_key: "56a95595c9641bf9674c5f75dfa1394f"
server_key_iv: "7af6d64782e34e3a004dab5798acbadf"
server_key_version: "encryptionkey@0"
I20260807 08:39:22.417255 8866 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 08:39:22.420574 8886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:22.422233 8866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:39:22.422413 8866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "6faf85ceaf204b4abbb9d3c102a5e1c6"
format_stamp: "Formatted at 2026-08-07 08:39:22 on dist-test-slave-h3ft"
server_key: "56a95595c9641bf9674c5f75dfa1394f"
server_key_iv: "7af6d64782e34e3a004dab5798acbadf"
server_key_version: "encryptionkey@0"
I20260807 08:39:22.422570 8866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:22.444285 8866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:22.447990 8866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:22.448230 8866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:22.457301 8866 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46151
I20260807 08:39:22.457326 8964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46151 every 8 connection(s)
I20260807 08:39:22.458561 8866 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:22.462085 8965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:22.462458 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 8866
I20260807 08:39:22.462642 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:22.463052 9912 external_mini_cluster.cc:1428] Setting key 7c837fbfe34e31d34d66755ff58b1365
I20260807 08:39:22.470782 8965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6: Bootstrap starting.
I20260807 08:39:22.473846 8965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6: Neither blocks nor log segments found. Creating new log.
Aug 07 08:39:22 dist-test-slave-h3ft krb5kdc[8845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 08:39:22.474856 8965 log.cc:824] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:22.477746 8965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6: No bootstrap required, opened a new log
I20260807 08:39:22.480659 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:22.464774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:51959 (local address 127.0.0.1:46151)
0807 08:39:22.465279 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:22.465301 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:22.465354 (+ 53us) server_negotiation.cc:407] Connection header received
0807 08:39:22.466239 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:22.466269 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:22.466772 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:22.467225 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:22.467995 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:22.469613 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:22.470617 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:22.471359 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:22.471530 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:22.474673 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:22.474683 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:22.474702 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:22.474741 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:22.477042 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:22.477626 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:22.477632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:22.477639 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:22.477794 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:22.478242 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:22.478245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:22.478247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:22.478716 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:22.478948 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:22.479351 (+ 403us) server_negotiation.cc:299] Negotiation successful
0807 08:39:22.479652 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":130,"threads_started":1}
I20260807 08:39:22.482414 8965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faf85ceaf204b4abbb9d3c102a5e1c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46151 } }
I20260807 08:39:22.482743 8965 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:22.482801 8965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6faf85ceaf204b4abbb9d3c102a5e1c6, State: Initialized, Role: FOLLOWER
I20260807 08:39:22.483404 8965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faf85ceaf204b4abbb9d3c102a5e1c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46151 } }
I20260807 08:39:22.483634 8965 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:22.483734 8965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:22.483862 8965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:22.484587 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:22.485409 8965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faf85ceaf204b4abbb9d3c102a5e1c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46151 } }
I20260807 08:39:22.485841 8965 leader_election.cc:302] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6faf85ceaf204b4abbb9d3c102a5e1c6; no voters:
I20260807 08:39:22.486270 8965 leader_election.cc:288] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:22.486730 8978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:22.486984 8978 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [term 1 LEADER]: Becoming Leader. State: Replica: 6faf85ceaf204b4abbb9d3c102a5e1c6, State: Running, Role: LEADER
I20260807 08:39:22.487394 8978 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faf85ceaf204b4abbb9d3c102a5e1c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46151 } }
I20260807 08:39:22.488109 8965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:22.489027 8978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6faf85ceaf204b4abbb9d3c102a5e1c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faf85ceaf204b4abbb9d3c102a5e1c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46151 } } }
I20260807 08:39:22.489212 8978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:22.489475 8978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6faf85ceaf204b4abbb9d3c102a5e1c6. Latest consensus state: current_term: 1 leader_uuid: "6faf85ceaf204b4abbb9d3c102a5e1c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faf85ceaf204b4abbb9d3c102a5e1c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46151 } } }
I20260807 08:39:22.489576 8978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6 [sys.catalog]: This master's current role is: LEADER
W20260807 08:39:22.494745 8995 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:39:22.494854 8995 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:39:22.494980 8987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:22.496172 8987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:22.497689 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:22.487467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:56721 (local address 127.0.0.1:46151)
0807 08:39:22.487927 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:22.487934 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:22.487953 (+ 19us) server_negotiation.cc:407] Connection header received
0807 08:39:22.488141 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:22.488144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:22.488215 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:22.488298 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:22.489414 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:22.490325 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:22.491525 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:22.491808 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:22.491916 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:22.493784 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:22.493788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:22.493792 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:22.493821 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:22.495703 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:22.496218 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:22.496224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:22.496251 (+ 27us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:22.496367 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:22.496714 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:22.496717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:22.496720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:22.496916 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:22.497099 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:22.497387 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 08:39:22.497492 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260807 08:39:22.503616 8987 catalog_manager.cc:1347] Generated new cluster ID: 6a77dfb7de0d48659124ba660b12b73c
I20260807 08:39:22.503716 8987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:22.536098 8987 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:22.537767 8987 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:22.545629 8987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6faf85ceaf204b4abbb9d3c102a5e1c6: Generated new TSK 0
I20260807 08:39:22.546545 8987 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:22.562254 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 8866
W20260807 08:39:22.575806 8842 connection.cc:535] client connection to 127.0.0.1:46151 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46151): Connection reset by peer (error 104)
2026-08-07T08:39:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1752 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:22 dist-test-slave-h3ft krb5kdc[9002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:22 dist-test-slave-h3ft krb5kdc[9002](info): set up 2 sockets
Aug 07 08:39:22 dist-test-slave-h3ft krb5kdc[9002](info): commencing operation
krb5kdc: starting...
W20260807 08:39:23.916782 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:24 dist-test-slave-h3ft krb5kdc[9002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:39:24.068141 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34903
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:24.204007 9017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:24.204393 9017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:24.204484 9017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:24.208663 9017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:24.208756 9017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:24.208796 9017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:24.208854 9017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:24.208886 9017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:24.214538 9017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34903
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:24.216061 9017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:24.217362 9017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:24.227438 9030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:24.227428 9028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:24.227479 9027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:24.227762 9017 server_base.cc:1034] running on GCE node
I20260807 08:39:24.228569 9017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:24.230046 9017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:24.231266 9017 hybrid_clock.cc:648] HybridClock initialized: now 1786091964231238 us; error 60 us; skew 500 ppm
Aug 07 08:39:24 dist-test-slave-h3ft krb5kdc[9002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091964, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:24.234786 9017 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:39:24.236135 9017 webserver.cc:460] Webserver started at http://127.0.0.1:35613/ using document root <none> and password file <none>
I20260807 08:39:24.236886 9017 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:24.237007 9017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:24.237315 9017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:24.240171 9017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "eddfdede223f4bfdae5d4441ad897394"
format_stamp: "Formatted at 2026-08-07 08:39:24 on dist-test-slave-h3ft"
server_key: "84f7f29f7cca1ac293fa8a357cebd601"
server_key_iv: "1dad2a8fc1d91cb1ce4ed99b1444fac8"
server_key_version: "encryptionkey@0"
I20260807 08:39:24.240923 9017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "eddfdede223f4bfdae5d4441ad897394"
format_stamp: "Formatted at 2026-08-07 08:39:24 on dist-test-slave-h3ft"
server_key: "84f7f29f7cca1ac293fa8a357cebd601"
server_key_iv: "1dad2a8fc1d91cb1ce4ed99b1444fac8"
server_key_version: "encryptionkey@0"
I20260807 08:39:24.245837 9017 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:39:24.249152 9041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:24.250648 9017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:39:24.250840 9017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "eddfdede223f4bfdae5d4441ad897394"
format_stamp: "Formatted at 2026-08-07 08:39:24 on dist-test-slave-h3ft"
server_key: "84f7f29f7cca1ac293fa8a357cebd601"
server_key_iv: "1dad2a8fc1d91cb1ce4ed99b1444fac8"
server_key_version: "encryptionkey@0"
I20260807 08:39:24.251087 9017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:24.275452 9017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:24.279434 9017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:24.279711 9017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:24.289966 9017 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34903
I20260807 08:39:24.289999 9115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34903 every 8 connection(s)
I20260807 08:39:24.291299 9017 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:24.294831 9116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:24.298383 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 9017
I20260807 08:39:24.298579 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:24.298934 9912 external_mini_cluster.cc:1428] Setting key aeddd8b556e030e8b9d0a01f56c1fc2b
I20260807 08:39:24.302039 9116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394: Bootstrap starting.
I20260807 08:39:24.304697 9116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:24.305598 9116 log.cc:824] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:24.307898 9116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394: No bootstrap required, opened a new log
I20260807 08:39:24.309255 9119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:24.300715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:53545 (local address 127.0.0.1:34903)
0807 08:39:24.301242 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:24.301259 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:24.301302 (+ 43us) server_negotiation.cc:407] Connection header received
0807 08:39:24.301998 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:24.302016 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:24.302462 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:24.302829 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:24.303384 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:24.304452 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:24.305465 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:24.306043 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:24.306235 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:24.307327 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:24.307332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:24.307346 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:24.307796 (+ 450us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:24.307865 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:24.307890 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:24.308053 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:24.308263 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 08:39:24.308418 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":164,"threads_started":1}
I20260807 08:39:24.312764 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:24.312536 9116 raft_consensus.cc:348] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eddfdede223f4bfdae5d4441ad897394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } }
I20260807 08:39:24.312867 9116 raft_consensus.cc:374] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:24.312969 9116 raft_consensus.cc:729] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eddfdede223f4bfdae5d4441ad897394, State: Initialized, Role: FOLLOWER
I20260807 08:39:24.313503 9116 consensus_queue.cc:260] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eddfdede223f4bfdae5d4441ad897394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } }
I20260807 08:39:24.313673 9116 raft_consensus.cc:388] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:24.313766 9116 raft_consensus.cc:482] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:24.313875 9116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:24.315135 9116 raft_consensus.cc:504] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eddfdede223f4bfdae5d4441ad897394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } }
I20260807 08:39:24.315604 9116 leader_election.cc:302] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eddfdede223f4bfdae5d4441ad897394; no voters:
I20260807 08:39:24.316010 9116 leader_election.cc:288] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:24.316174 9129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:24.316607 9129 raft_consensus.cc:686] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [term 1 LEADER]: Becoming Leader. State: Replica: eddfdede223f4bfdae5d4441ad897394, State: Running, Role: LEADER
I20260807 08:39:24.317152 9129 consensus_queue.cc:237] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eddfdede223f4bfdae5d4441ad897394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } }
I20260807 08:39:24.317662 9116 sys_catalog.cc:564] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:24.318739 9129 sys_catalog.cc:455] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eddfdede223f4bfdae5d4441ad897394. Latest consensus state: current_term: 1 leader_uuid: "eddfdede223f4bfdae5d4441ad897394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eddfdede223f4bfdae5d4441ad897394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } } }
I20260807 08:39:24.318919 9129 sys_catalog.cc:458] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:24.319406 9131 sys_catalog.cc:455] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eddfdede223f4bfdae5d4441ad897394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eddfdede223f4bfdae5d4441ad897394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } } }
I20260807 08:39:24.319521 9131 sys_catalog.cc:458] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:24.322359 9138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:24.323323 9119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:24.315478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:43767 (local address 127.0.0.1:34903)
0807 08:39:24.315635 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:24.315639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:24.315820 (+ 181us) server_negotiation.cc:407] Connection header received
0807 08:39:24.316084 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:24.316091 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:24.316167 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:24.316287 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:24.316663 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:24.317432 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:24.319688 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:24.320086 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:24.320184 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:24.322323 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:24.322329 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:24.322333 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:24.322348 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:24.322386 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:24.322408 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:24.322511 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:24.323102 (+ 591us) server_negotiation.cc:299] Negotiation successful
0807 08:39:24.323150 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 08:39:24.326419 9138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:24.333320 9138 catalog_manager.cc:1347] Generated new cluster ID: b637e874fddd4d46ae93cd47fe83fc0d
I20260807 08:39:24.333402 9138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:24.345625 9138 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:24.347543 9138 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:24.360973 9138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eddfdede223f4bfdae5d4441ad897394: Generated new TSK 0
I20260807 08:39:24.361999 9138 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:24.367375 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 9017
W20260807 08:39:24.379082 8999 connection.cc:535] client connection to 127.0.0.1:34903 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34903): Connection reset by peer (error 104)
2026-08-07T08:39:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1803 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:24 dist-test-slave-h3ft krb5kdc[9155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:24 dist-test-slave-h3ft krb5kdc[9155](info): set up 2 sockets
Aug 07 08:39:24 dist-test-slave-h3ft krb5kdc[9155](info): commencing operation
krb5kdc: starting...
W20260807 08:39:25.667733 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:25 dist-test-slave-h3ft krb5kdc[9155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:39:25.832082 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34591
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:25.967156 9171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:25.967543 9171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:25.967619 9171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:25.971760 9171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:25.971861 9171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:25.971901 9171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:25.971948 9171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:25.971982 9171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:25.977629 9171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34591
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:25.979054 9171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:25.980347 9171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:25.990355 9182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:25.990375 9177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:25.990525 9171 server_base.cc:1034] running on GCE node
W20260807 08:39:25.990360 9176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:25.991427 9171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:25.992816 9171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:25.994027 9171 hybrid_clock.cc:648] HybridClock initialized: now 1786091965994005 us; error 31 us; skew 500 ppm
Aug 07 08:39:25 dist-test-slave-h3ft krb5kdc[9155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091965, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:26.002328 9171 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:39:26.003573 9171 webserver.cc:460] Webserver started at http://127.0.0.1:41513/ using document root <none> and password file <none>
I20260807 08:39:26.004268 9171 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:26.004380 9171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:26.004681 9171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:26.007251 9171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "bb30e092fdab4b2ca847f6df6d58280e"
format_stamp: "Formatted at 2026-08-07 08:39:25 on dist-test-slave-h3ft"
server_key: "680804f36250929f2699b21f30cfb07c"
server_key_iv: "a2dfb0bb970ab2707518abd3792e3432"
server_key_version: "encryptionkey@0"
I20260807 08:39:26.007923 9171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "bb30e092fdab4b2ca847f6df6d58280e"
format_stamp: "Formatted at 2026-08-07 08:39:25 on dist-test-slave-h3ft"
server_key: "680804f36250929f2699b21f30cfb07c"
server_key_iv: "a2dfb0bb970ab2707518abd3792e3432"
server_key_version: "encryptionkey@0"
I20260807 08:39:26.012430 9171 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 08:39:26.015394 9192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:26.016891 9171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:39:26.017062 9171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "bb30e092fdab4b2ca847f6df6d58280e"
format_stamp: "Formatted at 2026-08-07 08:39:25 on dist-test-slave-h3ft"
server_key: "680804f36250929f2699b21f30cfb07c"
server_key_iv: "a2dfb0bb970ab2707518abd3792e3432"
server_key_version: "encryptionkey@0"
I20260807 08:39:26.017195 9171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:26.022560 9171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:26.026473 9171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:26.026713 9171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:26.035574 9171 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34591
I20260807 08:39:26.035598 9263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34591 every 8 connection(s)
I20260807 08:39:26.036789 9171 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:26.040370 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:26.041986 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 9171
I20260807 08:39:26.042173 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:26.042574 9912 external_mini_cluster.cc:1428] Setting key 42222ed9487ab8b50cb398351ae59a56
I20260807 08:39:26.048295 9265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e: Bootstrap starting.
I20260807 08:39:26.048971 9268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:26.044393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:54069 (local address 127.0.0.1:34591)
0807 08:39:26.044873 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:26.044902 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:26.044947 (+ 45us) server_negotiation.cc:407] Connection header received
0807 08:39:26.045663 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:26.045683 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:26.046014 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:26.046338 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:26.047384 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:26.047390 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:26.047402 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:26.047415 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:26.047482 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:26.047521 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:26.047656 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:26.047774 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 08:39:26.047996 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":147,"threads_started":1}
I20260807 08:39:26.051167 9265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:26.052037 9265 log.cc:824] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:26.052196 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:26.054778 9265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e: No bootstrap required, opened a new log
I20260807 08:39:26.056696 9268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:26.054876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:49683 (local address 127.0.0.1:34591)
0807 08:39:26.055036 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:26.055041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:26.055240 (+ 199us) server_negotiation.cc:407] Connection header received
0807 08:39:26.055348 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:26.055351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:26.055391 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:26.055472 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:26.056294 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:26.056298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:26.056301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:26.056313 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:26.056346 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:26.056362 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:26.056458 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:26.056510 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 08:39:26.056554 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 08:39:26.059542 9265 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb30e092fdab4b2ca847f6df6d58280e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34591 } }
I20260807 08:39:26.059823 9265 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:26.059859 9265 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb30e092fdab4b2ca847f6df6d58280e, State: Initialized, Role: FOLLOWER
I20260807 08:39:26.060287 9265 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb30e092fdab4b2ca847f6df6d58280e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34591 } }
I20260807 08:39:26.060420 9265 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:26.060467 9265 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:26.060547 9265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:26.061600 9265 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb30e092fdab4b2ca847f6df6d58280e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34591 } }
I20260807 08:39:26.061944 9265 leader_election.cc:302] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb30e092fdab4b2ca847f6df6d58280e; no voters:
I20260807 08:39:26.062264 9265 leader_election.cc:288] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:26.062422 9277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:26.062723 9277 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [term 1 LEADER]: Becoming Leader. State: Replica: bb30e092fdab4b2ca847f6df6d58280e, State: Running, Role: LEADER
I20260807 08:39:26.063197 9277 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb30e092fdab4b2ca847f6df6d58280e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34591 } }
I20260807 08:39:26.063871 9265 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:26.065403 9278 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb30e092fdab4b2ca847f6df6d58280e. Latest consensus state: current_term: 1 leader_uuid: "bb30e092fdab4b2ca847f6df6d58280e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb30e092fdab4b2ca847f6df6d58280e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34591 } } }
I20260807 08:39:26.065423 9279 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb30e092fdab4b2ca847f6df6d58280e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb30e092fdab4b2ca847f6df6d58280e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34591 } } }
I20260807 08:39:26.065639 9278 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:26.065657 9279 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:26.066684 9294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:26.070286 9294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:26.077559 9294 catalog_manager.cc:1347] Generated new cluster ID: 453cb7e33a05415b90663c039592f615
I20260807 08:39:26.077670 9294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:26.088389 9294 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:26.088855 9294 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:26.096194 9294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb30e092fdab4b2ca847f6df6d58280e: Generated new TSK 0
I20260807 08:39:26.097242 9294 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:26.103519 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 9171
2026-08-07T08:39:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1746 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:26 dist-test-slave-h3ft krb5kdc[9312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:26 dist-test-slave-h3ft krb5kdc[9312](info): set up 2 sockets
Aug 07 08:39:26 dist-test-slave-h3ft krb5kdc[9312](info): commencing operation
krb5kdc: starting...
W20260807 08:39:27.440012 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:27 dist-test-slave-h3ft krb5kdc[9312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:39:27.591372 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:33027
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:40481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:33027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:27.730358 9333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:27.730780 9333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:27.730867 9333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:27.735365 9333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:27.735514 9333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:27.735561 9333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:27.735606 9333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:27.735646 9333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:27.742264 9333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:40481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:33027
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:33027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:27.743867 9333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:27.745307 9333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:27.756261 9344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:27.756261 9340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:27.756261 9338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:27.756659 9333 server_base.cc:1034] running on GCE node
I20260807 08:39:27.757565 9333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:27.759138 9333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:27.760349 9333 hybrid_clock.cc:648] HybridClock initialized: now 1786091967760335 us; error 37 us; skew 500 ppm
Aug 07 08:39:27 dist-test-slave-h3ft krb5kdc[9312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091967, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:27.763761 9333 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:39:27.764930 9333 webserver.cc:460] Webserver started at http://127.9.174.62:35395/ using document root <none> and password file <none>
I20260807 08:39:27.765628 9333 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:27.765726 9333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:27.766016 9333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:27.768573 9333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "5c2a9a46943e49de9883bf3739e7f22c"
format_stamp: "Formatted at 2026-08-07 08:39:27 on dist-test-slave-h3ft"
server_key: "aea70abc86bedb37be0a3ebbc7ff7e65"
server_key_iv: "73437e806e7dd7d510997530cbb28a2a"
server_key_version: "encryptionkey@0"
I20260807 08:39:27.769243 9333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "5c2a9a46943e49de9883bf3739e7f22c"
format_stamp: "Formatted at 2026-08-07 08:39:27 on dist-test-slave-h3ft"
server_key: "aea70abc86bedb37be0a3ebbc7ff7e65"
server_key_iv: "73437e806e7dd7d510997530cbb28a2a"
server_key_version: "encryptionkey@0"
I20260807 08:39:27.773797 9333 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 08:39:27.776846 9355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:27.778359 9333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:39:27.778545 9333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "5c2a9a46943e49de9883bf3739e7f22c"
format_stamp: "Formatted at 2026-08-07 08:39:27 on dist-test-slave-h3ft"
server_key: "aea70abc86bedb37be0a3ebbc7ff7e65"
server_key_iv: "73437e806e7dd7d510997530cbb28a2a"
server_key_version: "encryptionkey@0"
I20260807 08:39:27.778690 9333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:27.819984 9333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:27.824678 9333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:27.824945 9333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:27.834568 9333 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:33027
I20260807 08:39:27.834590 9427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:33027 every 8 connection(s)
I20260807 08:39:27.835858 9333 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:27.839350 9428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:27.841941 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 9333
I20260807 08:39:27.842144 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:27.842506 9912 external_mini_cluster.cc:1428] Setting key 848d2096ac94f11d94201491edd5544f
I20260807 08:39:27.847721 9428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c: Bootstrap starting.
I20260807 08:39:27.851166 9428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:27.852324 9428 log.cc:824] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:39:27 dist-test-slave-h3ft krb5kdc[9312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:39:27.855269 9428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c: No bootstrap required, opened a new log
I20260807 08:39:27.860323 9428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2a9a46943e49de9883bf3739e7f22c" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33027 } }
I20260807 08:39:27.860618 9428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:27.860709 9428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c2a9a46943e49de9883bf3739e7f22c, State: Initialized, Role: FOLLOWER
I20260807 08:39:27.861297 9428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2a9a46943e49de9883bf3739e7f22c" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33027 } }
I20260807 08:39:27.861439 9428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:27.861498 9432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:27.844410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55881 (local address 127.9.174.62:33027)
0807 08:39:27.844964 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:27.844987 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:27.845043 (+ 56us) server_negotiation.cc:407] Connection header received
0807 08:39:27.846143 (+ 1100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:27.846170 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:27.846743 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:27.847211 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:27.848183 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:27.849849 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:27.850812 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:27.851692 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:27.851855 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:27.855365 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:27.855376 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:27.855405 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:27.855438 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:27.857367 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:27.858109 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:27.858114 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:27.858120 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:27.858253 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:27.858689 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:27.858692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:27.858694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:27.859354 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:27.859583 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:27.860006 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 08:39:27.860349 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":151,"threads_started":1}
I20260807 08:39:27.861534 9428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:27.861729 9428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:27.863075 9428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2a9a46943e49de9883bf3739e7f22c" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33027 } }
I20260807 08:39:27.863499 9428 leader_election.cc:302] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c2a9a46943e49de9883bf3739e7f22c; no voters:
I20260807 08:39:27.863971 9428 leader_election.cc:288] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:27.864095 9434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:27.864336 9434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [term 1 LEADER]: Becoming Leader. State: Replica: 5c2a9a46943e49de9883bf3739e7f22c, State: Running, Role: LEADER
I20260807 08:39:27.864760 9434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2a9a46943e49de9883bf3739e7f22c" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33027 } }
I20260807 08:39:27.865272 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:27.865653 9428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:27.866926 9435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c2a9a46943e49de9883bf3739e7f22c. Latest consensus state: current_term: 1 leader_uuid: "5c2a9a46943e49de9883bf3739e7f22c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2a9a46943e49de9883bf3739e7f22c" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33027 } } }
I20260807 08:39:27.867117 9435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:27.867537 9438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c2a9a46943e49de9883bf3739e7f22c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2a9a46943e49de9883bf3739e7f22c" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 33027 } } }
I20260807 08:39:27.867718 9438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:27.868673 9454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:27.872620 9454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:27.878497 9432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:27.869233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49881 (local address 127.9.174.62:33027)
0807 08:39:27.869373 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:27.869378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:27.869790 (+ 412us) server_negotiation.cc:407] Connection header received
0807 08:39:27.869955 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:27.869959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:27.870017 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:27.870110 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:27.870853 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:27.871499 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:27.872365 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:27.872639 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:27.872759 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:27.874358 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:27.874362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:27.874365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:27.874392 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:27.876253 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:27.876902 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:27.876906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:27.876908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:27.877002 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:27.877424 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:27.877429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:27.877434 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:27.877680 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:27.877838 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:27.878171 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 08:39:27.878289 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:39:27.880293 9454 catalog_manager.cc:1347] Generated new cluster ID: a0560627e88d48ae857751684af45d9d
I20260807 08:39:27.880367 9454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:27.897482 9454 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:27.898936 9454 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:27.910846 9454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c2a9a46943e49de9883bf3739e7f22c: Generated new TSK 0
I20260807 08:39:27.911991 9454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:27.922375 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 9333
W20260807 08:39:27.937218 9304 connection.cc:535] client connection to 127.9.174.62:33027 recv error: Network error: failed to read from TLS socket (remote: 127.9.174.62:33027): Connection reset by peer (error 104)
2026-08-07T08:39:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1811 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:27 dist-test-slave-h3ft krb5kdc[9471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:27 dist-test-slave-h3ft krb5kdc[9471](info): set up 2 sockets
Aug 07 08:39:27 dist-test-slave-h3ft krb5kdc[9471](info): commencing operation
krb5kdc: starting...
W20260807 08:39:29.198791 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:29 dist-test-slave-h3ft krb5kdc[9471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:39:29.342495 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:46035
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:34617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:46035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:29.475064 9497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:29.475487 9497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:29.475571 9497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:29.479935 9497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:29.480062 9497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:29.480103 9497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:29.480146 9497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:29.480187 9497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:29.486127 9497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:34617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:46035
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:46035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:29.487689 9497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:29.488993 9497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:29.499253 9504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:29.499253 9503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:29.499440 9497 server_base.cc:1034] running on GCE node
W20260807 08:39:29.499461 9506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:29.500252 9497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:29.501621 9497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:29.502799 9497 hybrid_clock.cc:648] HybridClock initialized: now 1786091969502771 us; error 41 us; skew 500 ppm
Aug 07 08:39:29 dist-test-slave-h3ft krb5kdc[9471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091969, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:29.511368 9497 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:39:29.512671 9497 webserver.cc:460] Webserver started at http://127.9.174.62:36003/ using document root <none> and password file <none>
I20260807 08:39:29.513414 9497 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:29.513520 9497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:29.513813 9497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:29.516528 9497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "a7f1aaf3e29c4a37bfa0a3615d3b075b"
format_stamp: "Formatted at 2026-08-07 08:39:29 on dist-test-slave-h3ft"
server_key: "53116d6ee474db6601e45482714ffcb9"
server_key_iv: "583370acc4982ab9014ddb77973430ac"
server_key_version: "encryptionkey@0"
I20260807 08:39:29.517217 9497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "a7f1aaf3e29c4a37bfa0a3615d3b075b"
format_stamp: "Formatted at 2026-08-07 08:39:29 on dist-test-slave-h3ft"
server_key: "53116d6ee474db6601e45482714ffcb9"
server_key_iv: "583370acc4982ab9014ddb77973430ac"
server_key_version: "encryptionkey@0"
I20260807 08:39:29.521875 9497 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 08:39:29.524845 9516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:29.526204 9497 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:39:29.526366 9497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "a7f1aaf3e29c4a37bfa0a3615d3b075b"
format_stamp: "Formatted at 2026-08-07 08:39:29 on dist-test-slave-h3ft"
server_key: "53116d6ee474db6601e45482714ffcb9"
server_key_iv: "583370acc4982ab9014ddb77973430ac"
server_key_version: "encryptionkey@0"
I20260807 08:39:29.526501 9497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:29.546296 9497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:29.550248 9497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:29.550518 9497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:29.559684 9497 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:46035
I20260807 08:39:29.559705 9591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:46035 every 8 connection(s)
I20260807 08:39:29.560876 9497 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:29.561707 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 9497
I20260807 08:39:29.561909 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:29.562268 9912 external_mini_cluster.cc:1428] Setting key 793b4744ce5ef14c2bce7ea85b65d693
I20260807 08:39:29.564742 9593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:29.572224 9593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b: Bootstrap starting.
I20260807 08:39:29.572688 9597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:29.564069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36505 (local address 127.9.174.62:46035)
0807 08:39:29.564570 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:29.564588 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:29.564631 (+ 43us) server_negotiation.cc:407] Connection header received
0807 08:39:29.565440 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:29.565459 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:29.565840 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:29.566158 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:29.566548 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:29.567718 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:29.568990 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:29.569478 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:29.569595 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:29.570597 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:29.570602 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:29.570614 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:29.571126 (+ 512us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:29.571193 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:29.571220 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:29.571328 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:29.571536 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 08:39:29.571718 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":175,"threads_started":1}
I20260807 08:39:29.575281 9593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:29.576253 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:29.576344 9593 log.cc:824] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:29.578967 9593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b: No bootstrap required, opened a new log
I20260807 08:39:29.583129 9597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:29.578743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45249 (local address 127.9.174.62:46035)
0807 08:39:29.578875 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:29.578880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:29.579245 (+ 365us) server_negotiation.cc:407] Connection header received
0807 08:39:29.579289 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:29.579292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:29.579332 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:29.579426 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:29.579732 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:29.580277 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:29.581187 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:29.581418 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:29.581533 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:29.582538 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:29.582544 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:29.582552 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:29.582565 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:29.582593 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:29.582605 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:29.582710 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:29.582873 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 08:39:29.582929 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 08:39:29.583408 9593 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f1aaf3e29c4a37bfa0a3615d3b075b" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 46035 } }
I20260807 08:39:29.583801 9593 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:29.583895 9593 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7f1aaf3e29c4a37bfa0a3615d3b075b, State: Initialized, Role: FOLLOWER
I20260807 08:39:29.584534 9593 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f1aaf3e29c4a37bfa0a3615d3b075b" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 46035 } }
I20260807 08:39:29.584704 9593 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:29.584779 9593 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:29.584897 9593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:29.586076 9593 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f1aaf3e29c4a37bfa0a3615d3b075b" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 46035 } }
I20260807 08:39:29.586500 9593 leader_election.cc:302] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7f1aaf3e29c4a37bfa0a3615d3b075b; no voters:
I20260807 08:39:29.586908 9593 leader_election.cc:288] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:29.587143 9606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:29.587416 9606 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [term 1 LEADER]: Becoming Leader. State: Replica: a7f1aaf3e29c4a37bfa0a3615d3b075b, State: Running, Role: LEADER
I20260807 08:39:29.587848 9606 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f1aaf3e29c4a37bfa0a3615d3b075b" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 46035 } }
I20260807 08:39:29.588634 9593 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:29.590107 9609 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7f1aaf3e29c4a37bfa0a3615d3b075b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f1aaf3e29c4a37bfa0a3615d3b075b" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 46035 } } }
I20260807 08:39:29.590127 9610 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7f1aaf3e29c4a37bfa0a3615d3b075b. Latest consensus state: current_term: 1 leader_uuid: "a7f1aaf3e29c4a37bfa0a3615d3b075b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f1aaf3e29c4a37bfa0a3615d3b075b" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 46035 } } }
I20260807 08:39:29.590318 9609 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:29.590348 9610 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:29.590791 9620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:29.594442 9620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:29.600792 9620 catalog_manager.cc:1347] Generated new cluster ID: e15d2eb4daef49eab72d841f553c953b
I20260807 08:39:29.600883 9620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:29.612241 9620 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:29.613418 9620 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:29.621641 9620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7f1aaf3e29c4a37bfa0a3615d3b075b: Generated new TSK 0
I20260807 08:39:29.622539 9620 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:29.628168 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 9497
W20260807 08:39:29.642328 9467 connection.cc:535] client connection to 127.9.174.62:46035 recv error: Network error: failed to read from TLS socket (remote: 127.9.174.62:46035): Connection reset by peer (error 104)
2026-08-07T08:39:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1704 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:29 dist-test-slave-h3ft krb5kdc[9637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:29 dist-test-slave-h3ft krb5kdc[9637](info): set up 2 sockets
Aug 07 08:39:29 dist-test-slave-h3ft krb5kdc[9637](info): commencing operation
krb5kdc: starting...
W20260807 08:39:30.932843 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:31 dist-test-slave-h3ft krb5kdc[9637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:39:31.078459 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:45841
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:33293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:45841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:31.215672 9652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:31.216106 9652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:31.216192 9652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:31.220432 9652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:31.220541 9652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:31.220585 9652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:31.220638 9652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:31.220678 9652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:31.226455 9652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:33293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:45841
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:45841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:31.227921 9652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:31.229269 9652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:31.240262 9660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:31.240262 9664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:31.240346 9658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:31.240576 9652 server_base.cc:1034] running on GCE node
I20260807 08:39:31.241489 9652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:31.243054 9652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:31.244283 9652 hybrid_clock.cc:648] HybridClock initialized: now 1786091971244243 us; error 57 us; skew 500 ppm
Aug 07 08:39:31 dist-test-slave-h3ft krb5kdc[9637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091971, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:31.252079 9652 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:39:31.253741 9652 webserver.cc:460] Webserver started at http://127.9.174.62:36039/ using document root <none> and password file <none>
I20260807 08:39:31.254545 9652 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:31.254622 9652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:31.254935 9652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:31.257789 9652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "c9413113eae54099b396cd5875f1af78"
format_stamp: "Formatted at 2026-08-07 08:39:31 on dist-test-slave-h3ft"
server_key: "69619565a6808393943e89de431afed2"
server_key_iv: "d814a438738a5a685bcb31c2c91f091d"
server_key_version: "encryptionkey@0"
I20260807 08:39:31.258545 9652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "c9413113eae54099b396cd5875f1af78"
format_stamp: "Formatted at 2026-08-07 08:39:31 on dist-test-slave-h3ft"
server_key: "69619565a6808393943e89de431afed2"
server_key_iv: "d814a438738a5a685bcb31c2c91f091d"
server_key_version: "encryptionkey@0"
I20260807 08:39:31.263586 9652 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 08:39:31.266868 9671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:31.268592 9652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:39:31.268769 9652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "c9413113eae54099b396cd5875f1af78"
format_stamp: "Formatted at 2026-08-07 08:39:31 on dist-test-slave-h3ft"
server_key: "69619565a6808393943e89de431afed2"
server_key_iv: "d814a438738a5a685bcb31c2c91f091d"
server_key_version: "encryptionkey@0"
I20260807 08:39:31.268950 9652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:31.274556 9652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:31.278120 9652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:31.278350 9652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:31.287217 9652 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:45841
I20260807 08:39:31.287251 9747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:45841 every 8 connection(s)
I20260807 08:39:31.288411 9652 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:31.291936 9748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:31.298341 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 9652
I20260807 08:39:31.298516 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:31.298859 9912 external_mini_cluster.cc:1428] Setting key 434bbf4f8caaa9b9be14a3f46930d4f8
I20260807 08:39:31.299229 9748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78: Bootstrap starting.
I20260807 08:39:31.302438 9748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:31.303506 9748 log.cc:824] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:31.305377 9752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:31.300633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47089 (local address 127.9.174.62:45841)
0807 08:39:31.301227 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:31.301245 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:31.301286 (+ 41us) server_negotiation.cc:407] Connection header received
0807 08:39:31.302078 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:31.302099 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:31.302449 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:31.302960 (+ 511us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:31.303866 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:31.303871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:31.303884 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:31.303894 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:31.303993 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:31.304031 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:31.304156 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:31.304307 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 08:39:31.304499 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":188,"threads_started":1}
I20260807 08:39:31.306056 9748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78: No bootstrap required, opened a new log
I20260807 08:39:31.308710 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:31.310477 9748 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9413113eae54099b396cd5875f1af78" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45841 } }
I20260807 08:39:31.310781 9748 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:31.310865 9748 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9413113eae54099b396cd5875f1af78, State: Initialized, Role: FOLLOWER
I20260807 08:39:31.311525 9748 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9413113eae54099b396cd5875f1af78" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45841 } }
I20260807 08:39:31.311702 9748 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:31.311779 9748 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:31.311901 9748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:31.313177 9748 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9413113eae54099b396cd5875f1af78" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45841 } }
I20260807 08:39:31.313632 9752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:31.311389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52823 (local address 127.9.174.62:45841)
0807 08:39:31.311558 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:31.311563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:31.311817 (+ 254us) server_negotiation.cc:407] Connection header received
0807 08:39:31.311972 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:31.311977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:31.312026 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:31.312117 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:31.313080 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:31.313086 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:31.313093 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:31.313110 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:31.313149 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:31.313167 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:31.313265 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:31.313392 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 08:39:31.313441 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 08:39:31.313604 9748 leader_election.cc:302] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9413113eae54099b396cd5875f1af78; no voters:
I20260807 08:39:31.314034 9748 leader_election.cc:288] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:31.314180 9761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:31.314407 9761 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [term 1 LEADER]: Becoming Leader. State: Replica: c9413113eae54099b396cd5875f1af78, State: Running, Role: LEADER
I20260807 08:39:31.314778 9761 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9413113eae54099b396cd5875f1af78" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45841 } }
I20260807 08:39:31.315677 9748 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:31.316635 9762 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9413113eae54099b396cd5875f1af78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9413113eae54099b396cd5875f1af78" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45841 } } }
I20260807 08:39:31.316771 9762 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:31.317104 9762 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9413113eae54099b396cd5875f1af78. Latest consensus state: current_term: 1 leader_uuid: "c9413113eae54099b396cd5875f1af78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9413113eae54099b396cd5875f1af78" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 45841 } } }
I20260807 08:39:31.317265 9762 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:31.317318 9771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:31.321185 9771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:31.327925 9771 catalog_manager.cc:1347] Generated new cluster ID: ca3f6a4cdf6547b8a62e7eb94308f1e6
I20260807 08:39:31.328013 9771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:31.342231 9771 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:31.342680 9771 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:31.347659 9771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9413113eae54099b396cd5875f1af78: Generated new TSK 0
I20260807 08:39:31.348646 9771 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:31.362039 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 9652
2026-08-07T08:39:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1739 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:31 dist-test-slave-h3ft krb5kdc[9790](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:31 dist-test-slave-h3ft krb5kdc[9790](info): set up 2 sockets
Aug 07 08:39:31 dist-test-slave-h3ft krb5kdc[9790](info): commencing operation
krb5kdc: starting...
W20260807 08:39:32.643986 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:32 dist-test-slave-h3ft krb5kdc[9790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:39:32.802515 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:39105
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:41547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:39105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:32.957258 9803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:32.957715 9803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:32.957815 9803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:32.962260 9803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:32.962405 9803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:32.962445 9803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:32.962483 9803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:32.962520 9803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:32.968710 9803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:41547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:39105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:39105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:32.970359 9803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:32.971760 9803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:32.981866 9811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:32.982051 9803 server_base.cc:1034] running on GCE node
W20260807 08:39:32.981846 9812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:32.981922 9816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:32.982967 9803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:32.984431 9803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:32.985644 9803 hybrid_clock.cc:648] HybridClock initialized: now 1786091972985627 us; error 34 us; skew 500 ppm
Aug 07 08:39:32 dist-test-slave-h3ft krb5kdc[9790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091972, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:32.989126 9803 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:39:32.990236 9803 webserver.cc:460] Webserver started at http://127.9.174.62:38187/ using document root <none> and password file <none>
I20260807 08:39:32.990890 9803 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:32.990955 9803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:32.991233 9803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:32.993754 9803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "712cbdf42b3c4fa89c0ee9d18668cc57"
format_stamp: "Formatted at 2026-08-07 08:39:32 on dist-test-slave-h3ft"
server_key: "248638dacceb11b45697b71690bedfae"
server_key_iv: "8c81e7931e5da7ed81f44383ed57abb5"
server_key_version: "encryptionkey@0"
I20260807 08:39:32.994370 9803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "712cbdf42b3c4fa89c0ee9d18668cc57"
format_stamp: "Formatted at 2026-08-07 08:39:32 on dist-test-slave-h3ft"
server_key: "248638dacceb11b45697b71690bedfae"
server_key_iv: "8c81e7931e5da7ed81f44383ed57abb5"
server_key_version: "encryptionkey@0"
I20260807 08:39:32.999269 9803 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 08:39:33.002339 9825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:33.003930 9803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:39:33.004067 9803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "712cbdf42b3c4fa89c0ee9d18668cc57"
format_stamp: "Formatted at 2026-08-07 08:39:32 on dist-test-slave-h3ft"
server_key: "248638dacceb11b45697b71690bedfae"
server_key_iv: "8c81e7931e5da7ed81f44383ed57abb5"
server_key_version: "encryptionkey@0"
I20260807 08:39:33.004177 9803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:33.040642 9803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:33.044610 9803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:33.044833 9803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:33.054448 9803 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:39105
I20260807 08:39:33.054483 9899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:39105 every 8 connection(s)
I20260807 08:39:33.055860 9803 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:33.059533 9901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:33.063513 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 9803
I20260807 08:39:33.063656 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:33.063984 9912 external_mini_cluster.cc:1428] Setting key 0eac12f0e6c13b9e7cbd9d3cba94f584
I20260807 08:39:33.067335 9901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57: Bootstrap starting.
I20260807 08:39:33.070783 9901 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:33.071755 9901 log.cc:824] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:33.074512 9901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57: No bootstrap required, opened a new log
Aug 07 08:39:33 dist-test-slave-h3ft krb5kdc[9790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:39:33.079475 9901 raft_consensus.cc:348] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712cbdf42b3c4fa89c0ee9d18668cc57" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39105 } }
I20260807 08:39:33.079890 9901 raft_consensus.cc:374] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:33.080013 9901 raft_consensus.cc:729] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 712cbdf42b3c4fa89c0ee9d18668cc57, State: Initialized, Role: FOLLOWER
I20260807 08:39:33.080651 9901 consensus_queue.cc:260] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712cbdf42b3c4fa89c0ee9d18668cc57" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39105 } }
I20260807 08:39:33.080850 9901 raft_consensus.cc:388] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:33.080935 9901 raft_consensus.cc:482] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:33.081063 9901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:33.082561 9901 raft_consensus.cc:504] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712cbdf42b3c4fa89c0ee9d18668cc57" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39105 } }
I20260807 08:39:33.082907 9906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:33.065865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57953 (local address 127.9.174.62:39105)
0807 08:39:33.066423 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:33.066447 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:33.066508 (+ 61us) server_negotiation.cc:407] Connection header received
0807 08:39:33.067479 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:33.067518 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:33.068017 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:33.068440 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:33.069265 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:33.070856 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:33.071863 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:33.072542 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:33.072702 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:33.075872 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:33.075879 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:33.075898 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:33.075934 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:33.078847 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:33.079461 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:33.079469 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:33.079476 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:33.079653 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:33.079999 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:33.080006 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:33.080011 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:33.080668 (+ 657us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:33.080916 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:33.081263 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 08:39:33.081648 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":154,"threads_started":1}
I20260807 08:39:33.083077 9901 leader_election.cc:302] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 712cbdf42b3c4fa89c0ee9d18668cc57; no voters:
I20260807 08:39:33.083518 9901 leader_election.cc:288] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:33.083669 9908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:33.083921 9908 raft_consensus.cc:686] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [term 1 LEADER]: Becoming Leader. State: Replica: 712cbdf42b3c4fa89c0ee9d18668cc57, State: Running, Role: LEADER
I20260807 08:39:33.084355 9908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712cbdf42b3c4fa89c0ee9d18668cc57" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39105 } }
I20260807 08:39:33.085178 9901 sys_catalog.cc:564] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:33.086418 9910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 712cbdf42b3c4fa89c0ee9d18668cc57. Latest consensus state: current_term: 1 leader_uuid: "712cbdf42b3c4fa89c0ee9d18668cc57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712cbdf42b3c4fa89c0ee9d18668cc57" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39105 } } }
I20260807 08:39:33.086622 9910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:33.086807 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:33.087105 9909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "712cbdf42b3c4fa89c0ee9d18668cc57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712cbdf42b3c4fa89c0ee9d18668cc57" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39105 } } }
I20260807 08:39:33.087199 9909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:33.087487 9926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:33.091810 9926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:33.099095 9906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:33.089648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55577 (local address 127.9.174.62:39105)
0807 08:39:33.089780 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:33.089786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:33.090066 (+ 280us) server_negotiation.cc:407] Connection header received
0807 08:39:33.090241 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:33.090244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:33.090303 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:33.090397 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:33.091240 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:33.091842 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:33.092790 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:33.093072 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:33.093190 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:33.095054 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:33.095058 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:33.095061 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:33.095090 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:33.097070 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:33.097525 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:33.097531 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:33.097533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:33.097626 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:33.097967 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:33.097971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:33.097973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:33.098249 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:33.098401 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:33.098699 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 08:39:33.098851 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:39:33.100189 9926 catalog_manager.cc:1347] Generated new cluster ID: 024db2cbea9f417586a1a76486a1beae
I20260807 08:39:33.100332 9926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:33.121346 9926 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:33.122761 9926 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:33.132550 9926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 712cbdf42b3c4fa89c0ee9d18668cc57: Generated new TSK 0
I20260807 08:39:33.133569 9926 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:33.149607 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 9803
2026-08-07T08:39:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1787 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:33 dist-test-slave-h3ft krb5kdc[9950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:33 dist-test-slave-h3ft krb5kdc[9950](info): set up 2 sockets
Aug 07 08:39:33 dist-test-slave-h3ft krb5kdc[9950](info): commencing operation
krb5kdc: starting...
W20260807 08:39:34.452265 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.001s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:34 dist-test-slave-h3ft krb5kdc[9950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:39:34.602547 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:39903
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:39903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:34.737206 9969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:34.737591 9969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:34.737640 9969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:34.742075 9969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:34.742162 9969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:34.742184 9969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:34.742202 9969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:34.742270 9969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:34.748235 9969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:39903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:39903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:34.749817 9969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:34.751215 9969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:34.761682 9978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:34.761950 9969 server_base.cc:1034] running on GCE node
W20260807 08:39:34.761682 9975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:34.761682 9976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:34.762961 9969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:34.764513 9969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:34.765753 9969 hybrid_clock.cc:648] HybridClock initialized: now 1786091974765729 us; error 40 us; skew 500 ppm
Aug 07 08:39:34 dist-test-slave-h3ft krb5kdc[9950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091974, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:34.769557 9969 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:39:34.770848 9969 webserver.cc:460] Webserver started at http://127.9.174.62:40541/ using document root <none> and password file <none>
I20260807 08:39:34.771657 9969 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:34.771771 9969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:34.772109 9969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:34.774726 9969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "e4c47548061e4033850b3e278709249f"
format_stamp: "Formatted at 2026-08-07 08:39:34 on dist-test-slave-h3ft"
server_key: "5e1beb659ff923efc901404f067de44b"
server_key_iv: "e0492891add1d923384b5cb4c5fbd92b"
server_key_version: "encryptionkey@0"
I20260807 08:39:34.775478 9969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "e4c47548061e4033850b3e278709249f"
format_stamp: "Formatted at 2026-08-07 08:39:34 on dist-test-slave-h3ft"
server_key: "5e1beb659ff923efc901404f067de44b"
server_key_iv: "e0492891add1d923384b5cb4c5fbd92b"
server_key_version: "encryptionkey@0"
I20260807 08:39:34.780207 9969 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:39:34.783222 9986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:34.784844 9969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 08:39:34.785025 9969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "e4c47548061e4033850b3e278709249f"
format_stamp: "Formatted at 2026-08-07 08:39:34 on dist-test-slave-h3ft"
server_key: "5e1beb659ff923efc901404f067de44b"
server_key_iv: "e0492891add1d923384b5cb4c5fbd92b"
server_key_version: "encryptionkey@0"
I20260807 08:39:34.785187 9969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:34.800263 9969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:34.804541 9969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:34.804819 9969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:34.814219 9969 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:39903
I20260807 08:39:34.814245 10059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:39903 every 8 connection(s)
I20260807 08:39:34.815616 9969 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:34.819486 10060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:34.822207 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 9969
I20260807 08:39:34.822381 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:34.822754 9912 external_mini_cluster.cc:1428] Setting key 7431c14fb5d309c5e32b6a652c57ce61
I20260807 08:39:34.828490 10060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f: Bootstrap starting.
I20260807 08:39:34.831552 10060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:34.832465 10060 log.cc:824] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:34.833997 10063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:34.824836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57349 (local address 127.9.174.62:39903)
0807 08:39:34.825459 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:34.825478 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:34.825525 (+ 47us) server_negotiation.cc:407] Connection header received
0807 08:39:34.826341 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:34.826366 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:34.826801 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:34.827337 (+ 536us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:34.827886 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:34.829100 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:34.830122 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:34.830664 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:34.830870 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:34.831965 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:34.831970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:34.831986 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:34.832457 (+ 471us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:34.832525 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:34.832549 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:34.832693 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:34.832872 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 08:39:34.833124 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":186,"threads_started":1}
I20260807 08:39:34.835129 10060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f: No bootstrap required, opened a new log
I20260807 08:39:34.837531 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:34.839759 10060 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c47548061e4033850b3e278709249f" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39903 } }
I20260807 08:39:34.840090 10060 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:34.840155 10060 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4c47548061e4033850b3e278709249f, State: Initialized, Role: FOLLOWER
I20260807 08:39:34.840687 10060 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c47548061e4033850b3e278709249f" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39903 } }
I20260807 08:39:34.840863 10060 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:34.840946 10060 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:34.841090 10060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:34.842412 10060 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c47548061e4033850b3e278709249f" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39903 } }
I20260807 08:39:34.842917 10060 leader_election.cc:302] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4c47548061e4033850b3e278709249f; no voters:
I20260807 08:39:34.843413 10060 leader_election.cc:288] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:34.843576 10074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:34.844029 10074 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [term 1 LEADER]: Becoming Leader. State: Replica: e4c47548061e4033850b3e278709249f, State: Running, Role: LEADER
I20260807 08:39:34.845166 10074 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c47548061e4033850b3e278709249f" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39903 } }
I20260807 08:39:34.845458 10060 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:34.845913 10063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:34.840177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54933 (local address 127.9.174.62:39903)
0807 08:39:34.840321 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:34.840327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:34.840567 (+ 240us) server_negotiation.cc:407] Connection header received
0807 08:39:34.840747 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:34.840750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:34.840798 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:34.840885 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:34.841195 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:34.841765 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:34.843257 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:34.843675 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:34.843789 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:34.844965 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:34.844972 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:34.844975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:34.844989 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:34.845023 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:34.845073 (+ 50us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:34.845145 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:34.845677 (+ 532us) server_negotiation.cc:299] Negotiation successful
0807 08:39:34.845752 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 08:39:34.849479 10076 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4c47548061e4033850b3e278709249f. Latest consensus state: current_term: 1 leader_uuid: "e4c47548061e4033850b3e278709249f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c47548061e4033850b3e278709249f" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39903 } } }
I20260807 08:39:34.849658 10076 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:34.850098 10094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:34.850318 10075 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4c47548061e4033850b3e278709249f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c47548061e4033850b3e278709249f" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 39903 } } }
I20260807 08:39:34.852193 10075 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:34.855589 10094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:34.863291 10094 catalog_manager.cc:1347] Generated new cluster ID: 5011445120d844a4bee96e3ee1a3e000
I20260807 08:39:34.863410 10094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:34.880915 10094 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:34.882583 10094 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:34.890923 10094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4c47548061e4033850b3e278709249f: Generated new TSK 0
I20260807 08:39:34.892177 10094 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:34.904704 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 9969
2026-08-07T08:39:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1754 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:34 dist-test-slave-h3ft krb5kdc[10101](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:34 dist-test-slave-h3ft krb5kdc[10101](info): set up 2 sockets
Aug 07 08:39:34 dist-test-slave-h3ft krb5kdc[10101](info): commencing operation
krb5kdc: starting...
W20260807 08:39:36.194518 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:36 dist-test-slave-h3ft krb5kdc[10101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:39:36.347919 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:35831
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:41305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:35831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:36.481530 10121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:36.481947 10121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:36.482028 10121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:36.486197 10121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:36.486306 10121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:36.486346 10121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:36.486387 10121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:36.486420 10121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:36.492256 10121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:41305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:35831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:35831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:36.493788 10121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:36.495139 10121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:36.505198 10128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:36.505185 10127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:36.505295 10121 server_base.cc:1034] running on GCE node
W20260807 08:39:36.505185 10131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:36.506214 10121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:36.507640 10121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:36.508828 10121 hybrid_clock.cc:648] HybridClock initialized: now 1786091976508805 us; error 35 us; skew 500 ppm
Aug 07 08:39:36 dist-test-slave-h3ft krb5kdc[10101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091976, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:36.515588 10121 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:39:36.516793 10121 webserver.cc:460] Webserver started at http://127.9.174.62:36947/ using document root <none> and password file <none>
I20260807 08:39:36.517513 10121 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:36.517580 10121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:36.517841 10121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:36.520458 10121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "ecb9c1c72f0b4a78aedd3401db353acf"
format_stamp: "Formatted at 2026-08-07 08:39:36 on dist-test-slave-h3ft"
server_key: "4c63d3dd52ea010589b4d0a8b2e7c1fa"
server_key_iv: "37abb14956757966c706cb5efa5b5e56"
server_key_version: "encryptionkey@0"
I20260807 08:39:36.521184 10121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "ecb9c1c72f0b4a78aedd3401db353acf"
format_stamp: "Formatted at 2026-08-07 08:39:36 on dist-test-slave-h3ft"
server_key: "4c63d3dd52ea010589b4d0a8b2e7c1fa"
server_key_iv: "37abb14956757966c706cb5efa5b5e56"
server_key_version: "encryptionkey@0"
I20260807 08:39:36.525748 10121 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 08:39:36.528913 10139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:36.530511 10121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 08:39:36.530679 10121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "ecb9c1c72f0b4a78aedd3401db353acf"
format_stamp: "Formatted at 2026-08-07 08:39:36 on dist-test-slave-h3ft"
server_key: "4c63d3dd52ea010589b4d0a8b2e7c1fa"
server_key_iv: "37abb14956757966c706cb5efa5b5e56"
server_key_version: "encryptionkey@0"
I20260807 08:39:36.530818 10121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:36.536402 10121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:36.540305 10121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:36.540571 10121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:36.549584 10121 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:35831
I20260807 08:39:36.549616 10213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:35831 every 8 connection(s)
I20260807 08:39:36.550760 10121 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:36.554266 10214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:36.557298 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 10121
I20260807 08:39:36.557475 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:36.557804 9912 external_mini_cluster.cc:1428] Setting key 6649f9f778c02b2fa39efa8298cdebd0
I20260807 08:39:36.561403 10214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf: Bootstrap starting.
I20260807 08:39:36.564210 10217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:36.559731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55303 (local address 127.9.174.62:35831)
0807 08:39:36.560266 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:36.560293 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:36.560343 (+ 50us) server_negotiation.cc:407] Connection header received
0807 08:39:36.561046 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:36.561065 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:36.561402 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:36.561768 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:36.562750 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:36.562756 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:36.562768 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:36.562781 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:36.562846 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:36.562867 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:36.563093 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:36.563162 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 08:39:36.563383 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":176,"threads_started":1}
I20260807 08:39:36.564479 10214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:36.565358 10214 log.cc:824] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:36.567438 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:36.567754 10214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf: No bootstrap required, opened a new log
I20260807 08:39:36.571940 10217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:36.569997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57899 (local address 127.9.174.62:35831)
0807 08:39:36.570141 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:36.570146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:36.570304 (+ 158us) server_negotiation.cc:407] Connection header received
0807 08:39:36.570413 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:36.570417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:36.570458 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:36.570559 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:36.571415 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:36.571422 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:36.571428 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:39:36.571443 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:36.571477 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:39:36.571490 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:36.571596 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:36.571745 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 08:39:36.571791 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 08:39:36.572649 10214 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb9c1c72f0b4a78aedd3401db353acf" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35831 } }
I20260807 08:39:36.572932 10214 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:36.573024 10214 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecb9c1c72f0b4a78aedd3401db353acf, State: Initialized, Role: FOLLOWER
I20260807 08:39:36.573583 10214 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb9c1c72f0b4a78aedd3401db353acf" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35831 } }
I20260807 08:39:36.573726 10214 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:36.573822 10214 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:36.573928 10214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:36.575132 10214 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb9c1c72f0b4a78aedd3401db353acf" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35831 } }
I20260807 08:39:36.575549 10214 leader_election.cc:302] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecb9c1c72f0b4a78aedd3401db353acf; no voters:
I20260807 08:39:36.575941 10214 leader_election.cc:288] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:36.576105 10225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:36.576377 10225 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [term 1 LEADER]: Becoming Leader. State: Replica: ecb9c1c72f0b4a78aedd3401db353acf, State: Running, Role: LEADER
I20260807 08:39:36.576769 10225 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb9c1c72f0b4a78aedd3401db353acf" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35831 } }
I20260807 08:39:36.577514 10214 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:36.579038 10227 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecb9c1c72f0b4a78aedd3401db353acf. Latest consensus state: current_term: 1 leader_uuid: "ecb9c1c72f0b4a78aedd3401db353acf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb9c1c72f0b4a78aedd3401db353acf" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35831 } } }
I20260807 08:39:36.579080 10226 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecb9c1c72f0b4a78aedd3401db353acf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb9c1c72f0b4a78aedd3401db353acf" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 35831 } } }
I20260807 08:39:36.579176 10227 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:36.579195 10226 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf [sys.catalog]: This master's current role is: LEADER
W20260807 08:39:36.583570 10245 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:39:36.583671 10245 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:39:36.583745 10237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:36.584545 10237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:36.592173 10237 catalog_manager.cc:1347] Generated new cluster ID: 8d482dcb180c407bbbb05254f29cd0fb
I20260807 08:39:36.592259 10237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:36.612886 10237 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:36.613394 10237 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:36.631878 10237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecb9c1c72f0b4a78aedd3401db353acf: Generated new TSK 0
I20260807 08:39:36.633167 10237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:36.642984 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 10121
2026-08-07T08:39:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1738 ms)
[----------] 18 tests from AuthTokenIssuingTest (31785 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:36 dist-test-slave-h3ft krb5kdc[10254](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:36 dist-test-slave-h3ft krb5kdc[10254](info): set up 2 sockets
Aug 07 08:39:36 dist-test-slave-h3ft krb5kdc[10254](info): commencing operation
krb5kdc: starting...
W20260807 08:39:37.929718 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:38 dist-test-slave-h3ft krb5kdc[10254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:39:38.082749 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:38725
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:33059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:38725,127.9.174.61:42549,127.9.174.60:45665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:38.213809 10276 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:39:38.214252 10276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:38.214327 10276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:38.214411 10276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:38.218766 10276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:38.218937 10276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:38.218990 10276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:38.219074 10276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:38.219112 10276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:38.224944 10276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:33059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:38725,127.9.174.61:42549,127.9.174.60:45665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:38725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:38.226369 10276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:38.227663 10276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:38.238382 10287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:38.238405 10276 server_base.cc:1034] running on GCE node
W20260807 08:39:38.238356 10285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:38.238356 10284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:38.239526 10276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:38.240892 10276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:38.242070 10276 hybrid_clock.cc:648] HybridClock initialized: now 1786091978242031 us; error 53 us; skew 500 ppm
Aug 07 08:39:38 dist-test-slave-h3ft krb5kdc[10254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091978, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:38.245617 10276 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:39:38.246846 10276 webserver.cc:460] Webserver started at http://127.9.174.62:45061/ using document root <none> and password file <none>
I20260807 08:39:38.247608 10276 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:38.247680 10276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:38.247953 10276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:38.250535 10276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "315ade3425744dbfb79cc9ca14e1bcdd"
format_stamp: "Formatted at 2026-08-07 08:39:38 on dist-test-slave-h3ft"
server_key: "bac8897964efb0d8defc920c209b8675"
server_key_iv: "fb8670bb6bc2a125eaef06feca3d3b30"
server_key_version: "encryptionkey@0"
I20260807 08:39:38.251272 10276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "315ade3425744dbfb79cc9ca14e1bcdd"
format_stamp: "Formatted at 2026-08-07 08:39:38 on dist-test-slave-h3ft"
server_key: "bac8897964efb0d8defc920c209b8675"
server_key_iv: "fb8670bb6bc2a125eaef06feca3d3b30"
server_key_version: "encryptionkey@0"
I20260807 08:39:38.255997 10276 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 08:39:38.258994 10295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:38.260568 10276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 08:39:38.260727 10276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "315ade3425744dbfb79cc9ca14e1bcdd"
format_stamp: "Formatted at 2026-08-07 08:39:38 on dist-test-slave-h3ft"
server_key: "bac8897964efb0d8defc920c209b8675"
server_key_iv: "fb8670bb6bc2a125eaef06feca3d3b30"
server_key_version: "encryptionkey@0"
I20260807 08:39:38.260854 10276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:38.278884 10276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:38.282924 10276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:38.283236 10276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:38.292068 10276 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:38725
I20260807 08:39:38.292457 10374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:38725 every 8 connection(s)
I20260807 08:39:38.293406 10276 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:38.297072 10377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:38.301908 10377 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:39:38.302666 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 10276
I20260807 08:39:38.302829 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:38.303196 9912 external_mini_cluster.cc:1428] Setting key 90e2a3534ec59af2f4d6b8260ab1ac5f
Aug 07 08:39:38 dist-test-slave-h3ft krb5kdc[10254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091978, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.174.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.61@KRBTEST.COM" created.
I20260807 08:39:38.321592 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.305283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:38725 (local address 127.0.0.1:58454)
0807 08:39:38.306070 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.306126 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:38.307228 (+ 1102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:38.308139 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:38.308144 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:38.308548 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:38.308959 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.308977 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.310929 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.310931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.311802 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.311812 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.312849 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.312852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.312977 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.313706 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:38.313725 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:38.315410 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:38.317772 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:38.317781 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:38.317791 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:38.318048 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:38.318525 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:38.318528 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:38.318529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:38.318651 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:38.319519 (+ 868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:38.319530 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:38.319734 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 08:39:38.319954 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 08:39:38.320436 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":128,"spinlock_wait_cycles":73856,"thread_start_us":219,"threads_started":1}
I20260807 08:39:38.321599 10379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.305283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58454 (local address 127.9.174.62:38725)
0807 08:39:38.305972 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:38.305998 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:38.306265 (+ 267us) server_negotiation.cc:407] Connection header received
0807 08:39:38.307594 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:38.307602 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:38.307895 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:38.308129 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:38.309132 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.310750 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.311941 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.312665 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.312924 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.315571 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:38.315578 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:38.315591 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:38.315615 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:38.317661 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.318234 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.318242 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.318244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.318421 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.318773 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.318780 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.318785 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.319381 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:38.319523 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:38.320079 (+ 556us) server_negotiation.cc:299] Negotiation successful
0807 08:39:38.320350 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":454,"spinlock_wait_cycles":7040,"thread_start_us":218,"threads_started":1}
I20260807 08:39:38.325019 10377 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:39:38.325984 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.325684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:42549 (local address 127.0.0.1:49274)
0807 08:39:38.325830 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.325912 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 08:39:38.326280 10301 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.174.61:42549: connect: Connection refused (error 111)
I20260807 08:39:38.327383 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.327165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:42549 (local address 127.0.0.1:49286)
0807 08:39:38.327280 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.327326 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 08:39:38.327593 10377 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.61:42549: Network error: Client connection negotiation failed: client connection to 127.9.174.61:42549: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.61.keytab.
Entry for principal kudu/127.9.174.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.61.keytab.
I20260807 08:39:38.339537 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.61:42549
--webserver_interface=127.9.174.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.61.keytab
--principal=kudu/127.9.174.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:33059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:38725,127.9.174.61:42549,127.9.174.60:45665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:39:38.376916 10377 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } attempt: 1
I20260807 08:39:38.378098 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.377697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:42549 (local address 127.0.0.1:49298)
0807 08:39:38.377905 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.377990 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 08:39:38.378968 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.378716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:42549 (local address 127.0.0.1:49302)
0807 08:39:38.378838 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.378895 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 08:39:38.379169 10377 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.61:42549: Network error: Client connection negotiation failed: client connection to 127.9.174.61:42549: connect: Connection refused (error 111)
I20260807 08:39:38.447348 10377 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } attempt: 2
I20260807 08:39:38.448561 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.448095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:42549 (local address 127.0.0.1:49310)
0807 08:39:38.448324 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.448419 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 08:39:38.449465 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.449178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:42549 (local address 127.0.0.1:49326)
0807 08:39:38.449317 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.449375 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 08:39:38.449680 10377 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.61:42549: Network error: Client connection negotiation failed: client connection to 127.9.174.61:42549: connect: Connection refused (error 111)
W20260807 08:39:38.479640 10386 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:39:38.480072 10386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:38.480139 10386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:38.480224 10386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:38.484498 10386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:38.484601 10386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:38.484640 10386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:38.484680 10386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:38.484715 10386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:38.490592 10386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:33059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:38725,127.9.174.61:42549,127.9.174.60:45665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.61.keytab
--principal=kudu/127.9.174.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.61:42549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.174.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:38.492093 10386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:38.493417 10386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:38.503440 10392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:38.503530 10396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:38.503664 10393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:38.503739 10386 server_base.cc:1034] running on GCE node
I20260807 08:39:38.504573 10386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:38.505857 10386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:38.507102 10386 hybrid_clock.cc:648] HybridClock initialized: now 1786091978507079 us; error 37 us; skew 500 ppm
Aug 07 08:39:38 dist-test-slave-h3ft krb5kdc[10254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091978, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:38.510548 10386 init.cc:377] Logged in from keytab as kudu/127.9.174.61@KRBTEST.COM (short username kudu)
I20260807 08:39:38.511703 10386 webserver.cc:460] Webserver started at http://127.9.174.61:41643/ using document root <none> and password file <none>
I20260807 08:39:38.512449 10386 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:38.512529 10386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:38.512853 10386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:38.515480 10386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/data/instance:
uuid: "9ecf3cc33aee4b099436fc482a254269"
format_stamp: "Formatted at 2026-08-07 08:39:38 on dist-test-slave-h3ft"
server_key: "f36f8a91433b22bd3619efaa7aad35e8"
server_key_iv: "1dfe11a34f5b4caab6ea67ce48003546"
server_key_version: "encryptionkey@0"
I20260807 08:39:38.516211 10386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/wal/instance:
uuid: "9ecf3cc33aee4b099436fc482a254269"
format_stamp: "Formatted at 2026-08-07 08:39:38 on dist-test-slave-h3ft"
server_key: "f36f8a91433b22bd3619efaa7aad35e8"
server_key_iv: "1dfe11a34f5b4caab6ea67ce48003546"
server_key_version: "encryptionkey@0"
I20260807 08:39:38.520898 10386 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 08:39:38.523757 10405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:38.525241 10386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:39:38.525410 10386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/wal
uuid: "9ecf3cc33aee4b099436fc482a254269"
format_stamp: "Formatted at 2026-08-07 08:39:38 on dist-test-slave-h3ft"
server_key: "f36f8a91433b22bd3619efaa7aad35e8"
server_key_iv: "1dfe11a34f5b4caab6ea67ce48003546"
server_key_version: "encryptionkey@0"
I20260807 08:39:38.525548 10386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:38.540874 10377 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } attempt: 3
I20260807 08:39:38.541941 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.541575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:42549 (local address 127.0.0.1:43244)
0807 08:39:38.541751 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.541837 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 08:39:38.542824 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.542559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:42549 (local address 127.0.0.1:43260)
0807 08:39:38.542695 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.542752 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 08:39:38.543042 10377 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.61:42549: Network error: Client connection negotiation failed: client connection to 127.9.174.61:42549: connect: Connection refused (error 111)
I20260807 08:39:38.553679 10386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:38.557298 10386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:38.557539 10386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:38.566219 10386 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.61:42549
I20260807 08:39:38.566561 10479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.61:42549 every 8 connection(s)
I20260807 08:39:38.567544 10386 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/data/info.pb
I20260807 08:39:38.569546 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 10386
I20260807 08:39:38.569702 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-1/wal/instance
I20260807 08:39:38.570120 9912 external_mini_cluster.cc:1428] Setting key d945a0bb691108971c33c58050871fc2
I20260807 08:39:38.571794 10482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:38.576982 10482 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.174.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.60@KRBTEST.COM" created.
Aug 07 08:39:38 dist-test-slave-h3ft krb5kdc[10254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091978, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.61@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:39:38.595113 10379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.579557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53860 (local address 127.9.174.62:38725)
0807 08:39:38.579773 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:38.579782 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:38.580874 (+ 1092us) server_negotiation.cc:407] Connection header received
0807 08:39:38.582133 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:38.582137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:38.582202 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:38.582277 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:38.583800 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.584737 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.586722 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.587089 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.587219 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.590305 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:38.590312 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:38.590318 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:38.590362 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:38.592444 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.593060 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.593067 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.593069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.593187 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.593571 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.593578 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.593581 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.593884 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:38.594013 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:38.594655 (+ 642us) server_negotiation.cc:299] Negotiation successful
0807 08:39:38.594825 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 08:39:38.596109 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.580005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:38725 (local address 127.0.0.1:53860)
0807 08:39:38.580662 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.580725 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:38.581803 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:38.582476 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:38.582483 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:38.582899 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:38.583633 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.583649 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.584894 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.584901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.586566 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.586583 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.587285 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.587290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.587511 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.588292 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:38.588312 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:38.590098 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:38.592608 (+ 2510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:38.592614 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:38.592631 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:38.592946 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:38.593296 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:38.593299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:38.593300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:38.593462 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:38.594002 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:38.594009 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:38.594336 (+ 327us) client_negotiation.cc:769] Sending connection context
0807 08:39:38.594543 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 08:39:38.594881 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":202,"threads_started":1}
I20260807 08:39:38.597906 10482 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:39:38 dist-test-slave-h3ft krb5kdc[10254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091978, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.61@KRBTEST.COM for kudu/127.9.174.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.60.keytab.
Entry for principal kudu/127.9.174.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.60.keytab.
I20260807 08:39:38.607828 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.60:45665
--webserver_interface=127.9.174.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.60.keytab
--principal=kudu/127.9.174.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:33059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:38725,127.9.174.61:42549,127.9.174.60:45665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:39:38.610554 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.598588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:42549 (local address 127.0.0.1:43264)
0807 08:39:38.598733 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.598756 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:38.598829 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:38.599638 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:38.599641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:38.599870 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:38.599994 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.600001 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.601176 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.601182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.601738 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.601764 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.602494 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.602498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.602587 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.603228 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:38.603242 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:38.604679 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:38.608207 (+ 3528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:38.608214 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:38.608219 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:38.608538 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:38.608910 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:38.608914 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:38.608916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:38.609036 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:38.609651 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:38.609657 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:38.609801 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 08:39:38.609928 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 08:39:38.610134 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 08:39:38.610579 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.598761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43264 (local address 127.9.174.61:42549)
0807 08:39:38.599230 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:38.599245 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:38.599269 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:39:38.599316 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:38.599321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:38.599509 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:38.599667 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:38.600109 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.601056 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.601874 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.602318 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.602496 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.604856 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:38.604868 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:38.604893 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:38.604918 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:38.608021 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.608684 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.608691 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.608693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.608796 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.609141 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.609143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.609145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.609533 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:38.609667 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:38.610032 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 08:39:38.610192 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":179,"threads_started":1}
I20260807 08:39:38.613507 10482 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:39:38.614540 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.614181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43668)
0807 08:39:38.614338 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.614450 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 08:39:38.614778 10406 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
I20260807 08:39:38.615828 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.615591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43670)
0807 08:39:38.615713 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.615765 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 08:39:38.616025 10482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:45665: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
I20260807 08:39:38.665328 10482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } attempt: 1
I20260807 08:39:38.666694 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.666224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43678)
0807 08:39:38.666466 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.666561 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 08:39:38.667629 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.667363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43694)
0807 08:39:38.667488 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.667547 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 08:39:38.667858 10482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:45665: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
I20260807 08:39:38.686286 10377 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } attempt: 4
Aug 07 08:39:38 dist-test-slave-h3ft krb5kdc[10254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091978, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for kudu/127.9.174.61@KRBTEST.COM
I20260807 08:39:38.698771 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.687292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:42549 (local address 127.0.0.1:43274)
0807 08:39:38.687578 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.687600 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:38.687742 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:38.688103 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:38.688105 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:38.688346 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:38.688497 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.688504 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.689694 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.689698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.690969 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.690985 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.691751 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.691754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.691869 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.692610 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:38.692624 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:38.694457 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:38.696998 (+ 2541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:38.697002 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:38.697004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:38.697277 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:38.697625 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:38.697628 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:38.697630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:38.697733 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:38.698216 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:38.698218 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:38.698334 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 08:39:38.698438 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 08:39:38.698591 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260807 08:39:38.698926 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.687329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43274 (local address 127.9.174.61:42549)
0807 08:39:38.687563 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:38.687569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:38.687698 (+ 129us) server_negotiation.cc:407] Connection header received
0807 08:39:38.687862 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:38.687866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:38.687942 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:38.688063 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:38.688648 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.689534 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.691202 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.691607 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.691729 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.694667 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:38.694672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:38.694677 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:38.694716 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:38.696841 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.697391 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.697395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.697398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.697506 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.697830 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.697833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.697835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.698104 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:38.698208 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:38.698553 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 08:39:38.698742 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 08:39:38.700459 10377 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:39:38.701438 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.701138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43698)
0807 08:39:38.701286 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.701353 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 08:39:38.702237 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.702004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43712)
0807 08:39:38.702123 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.702182 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 08:39:38.702414 10377 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:45665: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
I20260807 08:39:38.736099 10482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } attempt: 2
I20260807 08:39:38.738054 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.736949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43720)
0807 08:39:38.737177 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.737299 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 08:39:38.739210 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.738882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43722)
0807 08:39:38.739055 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.739119 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 08:39:38.739395 10482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:45665: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
W20260807 08:39:38.756269 10490 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:39:38.756649 10490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:38.756675 10490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:38.756716 10490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:38.760962 10490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:38.761054 10490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:38.761076 10490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:38.761093 10490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:38.761111 10490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:38.761627 10377 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } attempt: 1
I20260807 08:39:38.762880 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.762450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43738)
0807 08:39:38.762657 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.762760 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 08:39:38.763855 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.763587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43746)
0807 08:39:38.763718 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.763783 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 08:39:38.764038 10377 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:45665: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
I20260807 08:39:38.767638 10490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:33059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:38725,127.9.174.61:42549,127.9.174.60:45665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.60.keytab
--principal=kudu/127.9.174.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.60:45665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.174.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:38.769156 10490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:38.770463 10490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:38.781045 10499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:38.781013 10504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:38.781185 10501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:38.781558 10490 server_base.cc:1034] running on GCE node
I20260807 08:39:38.782295 10490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:38.783839 10490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:38.785079 10490 hybrid_clock.cc:648] HybridClock initialized: now 1786091978785041 us; error 55 us; skew 500 ppm
Aug 07 08:39:38 dist-test-slave-h3ft krb5kdc[10254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091978, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:38.788771 10490 init.cc:377] Logged in from keytab as kudu/127.9.174.60@KRBTEST.COM (short username kudu)
I20260807 08:39:38.789906 10490 webserver.cc:460] Webserver started at http://127.9.174.60:45895/ using document root <none> and password file <none>
I20260807 08:39:38.790577 10490 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:38.790642 10490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:38.790853 10490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:38.793406 10490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/data/instance:
uuid: "d7438d7f75e0454cb234375d374403a9"
format_stamp: "Formatted at 2026-08-07 08:39:38 on dist-test-slave-h3ft"
server_key: "d92a46e654353c60c92d214ff488d71e"
server_key_iv: "200c2786ca5666c53da974c59a57df25"
server_key_version: "encryptionkey@0"
I20260807 08:39:38.794042 10490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/wal/instance:
uuid: "d7438d7f75e0454cb234375d374403a9"
format_stamp: "Formatted at 2026-08-07 08:39:38 on dist-test-slave-h3ft"
server_key: "d92a46e654353c60c92d214ff488d71e"
server_key_iv: "200c2786ca5666c53da974c59a57df25"
server_key_version: "encryptionkey@0"
I20260807 08:39:38.798667 10490 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260807 08:39:38.801854 10512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:38.803568 10490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 08:39:38.803736 10490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/wal
uuid: "d7438d7f75e0454cb234375d374403a9"
format_stamp: "Formatted at 2026-08-07 08:39:38 on dist-test-slave-h3ft"
server_key: "d92a46e654353c60c92d214ff488d71e"
server_key_iv: "200c2786ca5666c53da974c59a57df25"
server_key_version: "encryptionkey@0"
I20260807 08:39:38.803918 10490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:38.820231 10490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:38.824368 10490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:38.824635 10490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:38.830549 10482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } attempt: 3
I20260807 08:39:38.831245 10377 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } attempt: 2
I20260807 08:39:38.831717 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.831287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43760)
0807 08:39:38.831511 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.831590 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 08:39:38.832175 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.831743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43766)
0807 08:39:38.832012 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.832076 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
I20260807 08:39:38.833055 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.832656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43776)
0807 08:39:38.832913 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.832974 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
I20260807 08:39:38.833055 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.832764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43784)
0807 08:39:38.832888 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.832973 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 08:39:38.833235 10482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:45665: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
W20260807 08:39:38.833254 10377 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:45665: Network error: Client connection negotiation failed: client connection to 127.9.174.60:45665: connect: Connection refused (error 111)
I20260807 08:39:38.834386 10490 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.60:45665
I20260807 08:39:38.834748 10580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.60:45665 every 8 connection(s)
I20260807 08:39:38.835734 10490 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/data/info.pb
I20260807 08:39:38.839360 10581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:38.839748 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 10490
I20260807 08:39:38.839912 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091655732880-9912-0/minicluster-data/master-2/wal/instance
I20260807 08:39:38.840323 9912 external_mini_cluster.cc:1428] Setting key f3006ccc7e1f164ae3070b65dea2fd34
I20260807 08:39:38.846427 10581 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:39:38 dist-test-slave-h3ft krb5kdc[10254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.60@KRBTEST.COM
Aug 07 08:39:38 dist-test-slave-h3ft krb5kdc[10254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091978, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.60@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:39:38.859305 10586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.842061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43800 (local address 127.9.174.60:45665)
0807 08:39:38.842686 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:38.842713 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:38.842769 (+ 56us) server_negotiation.cc:407] Connection header received
0807 08:39:38.844084 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:38.844117 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:38.844656 (+ 539us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:38.845120 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:38.846384 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.848428 (+ 2044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.849745 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.850564 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.850784 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.854016 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:38.854024 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:38.854039 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:38.854077 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:38.856116 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.856581 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.856587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.856600 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.856711 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.857028 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.857031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.857033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.857472 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:38.857689 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:38.858025 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 08:39:38.858327 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":158,"threads_started":1}
I20260807 08:39:38.859814 10379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.849027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53874 (local address 127.9.174.62:38725)
0807 08:39:38.849188 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:38.849193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:38.849860 (+ 667us) server_negotiation.cc:407] Connection header received
0807 08:39:38.850017 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:38.850020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:38.850086 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:38.850181 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:38.851272 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.852025 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.852968 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.853209 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.853323 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.856207 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:38.856211 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:38.856214 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:38.856252 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:38.857831 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.858382 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.858386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.858388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.858477 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.858767 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.858772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.858774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.858988 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:38.859169 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:38.859569 (+ 400us) server_negotiation.cc:299] Negotiation successful
0807 08:39:38.859685 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 08:39:38.860298 10587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.848834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:38725 (local address 127.0.0.1:53874)
0807 08:39:38.849680 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.849745 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:38.849902 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:38.850286 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:38.850293 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:38.850878 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:38.851130 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.851138 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.852178 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.852181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.852836 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.852846 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.853386 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.853390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.853512 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.854108 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:38.854125 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:38.856052 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:38.857968 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:38.857977 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:38.857983 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:38.858297 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:38.858576 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:38.858580 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:38.858583 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:38.858674 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:38.859166 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:38.859173 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:38.859298 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 08:39:38.860000 (+ 702us) client_negotiation.cc:240] Negotiation successful
0807 08:39:38.860171 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":213,"threads_started":1}
I20260807 08:39:38.861536 10581 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:39:38 dist-test-slave-h3ft krb5kdc[10254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091978, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.60@KRBTEST.COM for kudu/127.9.174.61@KRBTEST.COM
Aug 07 08:39:38 dist-test-slave-h3ft krb5kdc[10254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:39:38.871188 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.862474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.9.174.61:42549)
0807 08:39:38.862618 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:38.862622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:38.862634 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:39:38.862679 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:38.862681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:38.862724 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:38.862820 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:38.863282 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.863794 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.864568 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.864790 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.864916 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.867416 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:38.867423 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:38.867430 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:38.867470 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:38.869221 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.869763 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.869766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.869767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.869848 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.870209 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.870214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.870216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.870438 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:38.870559 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:38.870844 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 08:39:38.870990 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 08:39:38.871188 10587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.862124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:42549 (local address 127.0.0.1:43290)
0807 08:39:38.862241 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.862263 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:38.862342 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:38.862834 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:38.862837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:38.863033 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:38.863151 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.863157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.863918 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.863922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.864446 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.864454 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.864909 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.864913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.865019 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.865556 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:38.865573 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:38.867239 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:38.869365 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:38.869368 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:38.869371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:38.869653 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:38.869953 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:38.869957 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:38.869958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:38.870104 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:38.870547 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:38.870551 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:38.870667 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 08:39:38.870781 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 08:39:38.870943 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 08:39:38.872092 10379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.862005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53890 (local address 127.9.174.62:38725)
0807 08:39:38.862121 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:38.862126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:38.862167 (+ 41us) server_negotiation.cc:407] Connection header received
0807 08:39:38.862320 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:38.862324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:38.862374 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:38.862446 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:38.863233 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.863747 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.864627 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.864939 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.865061 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.867954 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:38.867958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:38.867961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:38.867998 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:38.869823 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.870285 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.870289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.870291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.870390 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.870683 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.870686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.870688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.870890 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:38.871210 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:38.871788 (+ 578us) server_negotiation.cc:299] Negotiation successful
0807 08:39:38.871925 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:39:38.872079 10581 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:39:38 dist-test-slave-h3ft krb5kdc[10254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091978, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.60@KRBTEST.COM for kudu/127.9.174.60@KRBTEST.COM
Aug 07 08:39:38 dist-test-slave-h3ft krb5kdc[10254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.61@KRBTEST.COM
I20260807 08:39:38.882138 10586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.872764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43810 (local address 127.9.174.60:45665)
0807 08:39:38.872868 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:38.872873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:38.872886 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:39:38.873008 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:38.873011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:38.873055 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:38.873128 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:38.873624 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.874507 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.875755 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.876124 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.876262 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.878842 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:38.878846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:38.878848 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:38.878872 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:38.880505 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.880908 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.880912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.880914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.881004 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.881275 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.881279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.881281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.881471 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:38.881579 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:38.881824 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 08:39:38.881991 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":41}
I20260807 08:39:38.882138 10587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.872675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43810)
0807 08:39:38.872809 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.872826 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:38.872909 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:38.873173 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:38.873176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:38.873356 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:38.873485 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.873494 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.874657 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.874660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.875582 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.875593 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.876370 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.876373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.876482 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.877027 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:38.877045 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:38.878710 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:38.880616 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:38.880619 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:38.880621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:38.880810 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:38.881096 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:38.881101 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:38.881103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:38.881186 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:38.881567 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:38.881570 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:38.881658 (+ 88us) client_negotiation.cc:769] Sending connection context
0807 08:39:38.881768 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 08:39:38.881920 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 08:39:38.883972 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.873590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43294 (local address 127.9.174.61:42549)
0807 08:39:38.873701 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:38.873704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:38.873716 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:39:38.873784 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:38.873786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:38.873822 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:38.873883 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:38.874640 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.875389 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.876510 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.876839 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.876940 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.880367 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:38.880370 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:38.880372 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:38.880394 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:38.881946 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.882482 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.882486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.882487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.882592 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.882891 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.882896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.882898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.883189 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:38.883593 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:38.883678 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 08:39:38.883805 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:39:38.887852 10581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9: Bootstrap starting.
I20260807 08:39:38.888054 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:38.889617 10333 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "315ade3425744dbfb79cc9ca14e1bcdd"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53890
I20260807 08:39:38.891434 10581 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:38.892498 10581 log.cc:824] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:38.895349 10581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9: No bootstrap required, opened a new log
I20260807 08:39:38.900372 10581 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } }
I20260807 08:39:38.900756 10581 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7438d7f75e0454cb234375d374403a9, State: Initialized, Role: FOLLOWER
I20260807 08:39:38.901175 10581 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } }
I20260807 08:39:38.901894 10591 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } } }
I20260807 08:39:38.902086 10591 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:39:38.902971 10581 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 08:39:38.909526 10604 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:39:38.909631 10604 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:39:38.908002 10490 master_runner.cc:186] Error getting master registration for 127.9.174.62:38725: OK, instance_id { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" instance_seqno: 1786091978289577 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:39:38.911490 10490 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:39:38.933432 10377 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } attempt: 3
Aug 07 08:39:38 dist-test-slave-h3ft krb5kdc[10254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091978, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for kudu/127.9.174.60@KRBTEST.COM
I20260807 08:39:38.944643 10586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.934610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43820 (local address 127.9.174.60:45665)
0807 08:39:38.934815 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:38.934841 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:38.934873 (+ 32us) server_negotiation.cc:407] Connection header received
0807 08:39:38.934940 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:38.934943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:38.935052 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:38.935151 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:38.935810 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.936429 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.937352 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.937613 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.937759 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.940607 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:38.940611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:38.940613 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:38.940649 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:38.942472 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.943045 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.943052 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.943059 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.943156 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.943504 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.943508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.943510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.943820 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:38.943952 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:38.944270 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 08:39:38.944431 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 08:39:38.944643 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.934276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43820)
0807 08:39:38.934492 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.934511 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:38.934635 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:38.935206 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:38.935210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:38.935504 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:38.935643 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.935650 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.936563 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.936569 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.937197 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.937206 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.937767 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.937771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.937872 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.938594 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:38.938609 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:38.940404 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:38.942617 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:38.942621 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:38.942624 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:38.942898 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:38.943274 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:38.943278 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:38.943280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:38.943395 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:38.943940 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:38.943943 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:38.944075 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 08:39:38.944207 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 08:39:38.944412 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 08:39:38.949826 10377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd: Bootstrap starting.
I20260807 08:39:38.953284 10377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:38.954530 10377 log.cc:824] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:38.957082 10377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd: No bootstrap required, opened a new log
I20260807 08:39:38.960289 10377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } }
I20260807 08:39:38.960594 10377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315ade3425744dbfb79cc9ca14e1bcdd, State: Initialized, Role: FOLLOWER
I20260807 08:39:38.961030 10377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } }
I20260807 08:39:38.961789 10607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } } }
I20260807 08:39:38.961951 10607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:39:38.962823 10377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:38.962924 10333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 08:39:38.963119 10333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:38.964793 10333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } }
I20260807 08:39:38.966295 10333 leader_election.cc:288] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [CANDIDATE]: Term 1 election: Requested vote from peers 9ecf3cc33aee4b099436fc482a254269 (127.9.174.61:42549), d7438d7f75e0454cb234375d374403a9 (127.9.174.60:45665)
I20260807 08:39:38.967372 10546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d7438d7f75e0454cb234375d374403a9"
I20260807 08:39:38.967751 10546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:38.969522 10546 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315ade3425744dbfb79cc9ca14e1bcdd in term 1.
I20260807 08:39:38.969503 10276 master_runner.cc:186] Error getting master registration for 127.9.174.61:42549: OK, instance_id { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" instance_seqno: 1786091978563707 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:39:38.974022 10276 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 08:39:38.974179 10622 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:39:38.974272 10622 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:39:38.974273 10297 leader_election.cc:302] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315ade3425744dbfb79cc9ca14e1bcdd, d7438d7f75e0454cb234375d374403a9; no voters:
I20260807 08:39:38.974627 10607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:38.974866 10607 raft_consensus.cc:686] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [term 1 LEADER]: Becoming Leader. State: Replica: 315ade3425744dbfb79cc9ca14e1bcdd, State: Running, Role: LEADER
I20260807 08:39:38.975524 10607 consensus_queue.cc:237] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } }
I20260807 08:39:38.976388 10482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } attempt: 4
I20260807 08:39:38.977900 10607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 315ade3425744dbfb79cc9ca14e1bcdd. Latest consensus state: current_term: 1 leader_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } } }
I20260807 08:39:38.978039 10607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:38.980226 10636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:38.986443 10636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:38.987239 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.972871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43308 (local address 127.9.174.61:42549)
0807 08:39:38.973033 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:38.973039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:38.976392 (+ 3353us) server_negotiation.cc:407] Connection header received
0807 08:39:38.976614 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:38.976618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:38.976681 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:38.976737 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:38.977900 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.978769 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.980273 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.980608 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.980699 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.982975 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:38.982979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:38.982983 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:38.983041 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:38.985015 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.985592 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.985596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.985598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.985701 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.986032 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.986035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.986037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.986304 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:38.986450 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:38.986847 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 08:39:38.987042 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 08:39:38.996659 10546 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9 [term 1 FOLLOWER]: Refusing update from remote peer 315ade3425744dbfb79cc9ca14e1bcdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:39:38.997747 10629 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:39:39.001760 10586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.975752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43824 (local address 127.9.174.60:45665)
0807 08:39:38.975887 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:38.975892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:38.987438 (+ 11546us) server_negotiation.cc:407] Connection header received
0807 08:39:38.987662 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:38.987666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:38.987739 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:38.987808 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:38.989006 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.989836 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.991211 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.991543 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.991647 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:38.995142 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:38.995147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:38.995150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:38.995183 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:38.998351 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.998937 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.998942 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.998944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:38.999080 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:38.999818 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:38.999821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:38.999823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:39.000090 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:39.000222 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:39.001417 (+ 1195us) server_negotiation.cc:299] Negotiation successful
0807 08:39:39.001569 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 08:39:39.003824 10591 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 315ade3425744dbfb79cc9ca14e1bcdd. Latest consensus state: current_term: 1 leader_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } } }
I20260807 08:39:39.004011 10591 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9 [sys.catalog]: This master's current role is: FOLLOWER
Aug 07 08:39:39 dist-test-slave-h3ft krb5kdc[10254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091978, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.61@KRBTEST.COM for kudu/127.9.174.60@KRBTEST.COM
I20260807 08:39:39.008903 10379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.974996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53898 (local address 127.9.174.62:38725)
0807 08:39:38.975142 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:38.975147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:38.987366 (+ 12219us) server_negotiation.cc:407] Connection header received
0807 08:39:38.987642 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:38.987646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:38.987721 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:38.987785 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:38.988847 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.989671 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.990927 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.991287 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.991388 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:39.004292 (+ 12904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:39.004297 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:39.004299 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:39.004342 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:39.006328 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:39.006912 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:39.006917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:39.006922 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:39.007055 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:39.007391 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:39.007410 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:39.007412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:39.007692 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:39.007839 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:39.008551 (+ 712us) server_negotiation.cc:299] Negotiation successful
0807 08:39:39.008704 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:39:39.009608 10629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } } }
I20260807 08:39:39.009765 10629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:39.009966 10638 mvcc.cc:205] Tried to move back new op lower bound from 7315832745959272448 to 7315832745888317440. Current Snapshot: MvccSnapshot[applied={T|T < 7315832745959272448}]
I20260807 08:39:39.011148 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.976824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:45665 (local address 127.0.0.1:43830)
0807 08:39:38.977118 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:38.977133 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:38.977221 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:38.992205 (+ 14984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:38.992210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:38.992455 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:38.992610 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.992619 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.995858 (+ 3239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.995875 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:38.996728 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:38.996742 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:39.003108 (+ 6366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:39.003112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:39.003229 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:39.003961 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:39.003981 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:39.006024 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:39.009079 (+ 3055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:39.009083 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:39.009085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:39.009408 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:39.009795 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:39.009799 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:39.009801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:39.009921 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:39.010469 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:39.010473 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:39.010617 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 08:39:39.010725 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 08:39:39.010911 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209}
I20260807 08:39:39.012189 10591 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } } }
I20260807 08:39:39.012341 10591 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:39:39.012807 10640 mvcc.cc:205] Tried to move back new op lower bound from 7315832745959272448 to 7315832745888317440. Current Snapshot: MvccSnapshot[applied={T|T < 7315832745959272448}]
I20260807 08:39:39.013237 10634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:38.977399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43830 (local address 127.9.174.60:45665)
0807 08:39:38.991433 (+ 14034us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:38.991447 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:38.991464 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:39:38.991508 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:38.991511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:38.991574 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:38.993964 (+ 2390us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:38.994090 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:38.994915 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:38.999451 (+ 4536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:39.000815 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:39.006194 (+ 5379us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:39.006335 (+ 141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:39.006339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:39.006342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:39.006360 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:39.008918 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:39.009562 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:39.009567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:39.009570 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:39.009670 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:39.010062 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:39.010067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:39.010070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:39.010322 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:39.012033 (+ 1711us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:39.012121 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 08:39:39.012907 (+ 786us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13910,"thread_start_us":119,"threads_started":1}
I20260807 08:39:39.017557 10482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269: Bootstrap starting.
I20260807 08:39:39.017828 10636 catalog_manager.cc:1347] Generated new cluster ID: 9dbb78ad3fdd4b248ee86ca0d1138f01
I20260807 08:39:39.017903 10636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:39.031319 10482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:39.032660 10482 log.cc:824] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:39.048480 10482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269: No bootstrap required, opened a new log
I20260807 08:39:39.060250 10482 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } }
I20260807 08:39:39.060907 10482 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecf3cc33aee4b099436fc482a254269, State: Initialized, Role: FOLLOWER
I20260807 08:39:39.064723 10482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } }
I20260807 08:39:39.068819 10438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:39.071080 10439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9ecf3cc33aee4b099436fc482a254269"
I20260807 08:39:39.071589 10438 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269 [term 1 FOLLOWER]: Refusing update from remote peer 315ade3425744dbfb79cc9ca14e1bcdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:39:39.072530 10607 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:39:39.077706 10439 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315ade3425744dbfb79cc9ca14e1bcdd in term 1.
I20260807 08:39:39.078754 10482 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:39.079631 10636 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:39.081164 10636 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:39.092034 10646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } } }
I20260807 08:39:39.092444 10646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:39:39.092643 10646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } } }
I20260807 08:39:39.092808 10646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:39:39.092751 10629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } } }
I20260807 08:39:39.092919 10629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:39.093051 10671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 315ade3425744dbfb79cc9ca14e1bcdd. Latest consensus state: current_term: 1 leader_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315ade3425744dbfb79cc9ca14e1bcdd" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 38725 } } peers { permanent_uuid: "9ecf3cc33aee4b099436fc482a254269" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 42549 } } peers { permanent_uuid: "d7438d7f75e0454cb234375d374403a9" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 45665 } } }
I20260807 08:39:39.093226 10671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 08:39:39.097307 10672 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:39:39.097532 10672 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:39:39.100378 10636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 315ade3425744dbfb79cc9ca14e1bcdd: Generated new TSK 0
I20260807 08:39:39.101534 10636 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:39.115617 10657 mvcc.cc:205] Tried to move back new op lower bound from 7315832746279817216 to 7315832745888317440. Current Snapshot: MvccSnapshot[applied={T|T < 7315832745959272448}]
W20260807 08:39:39.146251 10489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:39.142289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43316 (local address 127.9.174.61:42549)
0807 08:39:39.142454 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:39.142459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:39.144566 (+ 2107us) server_negotiation.cc:407] Connection header received
0807 08:39:39.145096 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:39.145102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:39.145171 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:39.145231 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:39.146042 (+ 811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43316: BlockingRecv error: recv got EOF from 127.0.0.1:43316 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 08:39:39.148183 10379 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:39.143400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53910 (local address 127.9.174.62:38725)
0807 08:39:39.143565 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:39.143570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:39.147146 (+ 3576us) server_negotiation.cc:407] Connection header received
0807 08:39:39.147334 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:39.147337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:39.147402 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:39.147458 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:39.148028 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53910: BlockingRecv error: recv got EOF from 127.0.0.1:53910 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 08:39:39.150094 10634 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:39.144232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43846 (local address 127.9.174.60:45665)
0807 08:39:39.144430 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:39.144435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:39.148817 (+ 4382us) server_negotiation.cc:407] Connection header received
0807 08:39:39.149144 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:39.149148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:39.149230 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:39.149297 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:39.149930 (+ 633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43846: BlockingRecv error: recv got EOF from 127.0.0.1:43846 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 08:39:39.156095 10489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:39.153173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43320 (local address 127.9.174.61:42549)
0807 08:39:39.153334 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:39.153340 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:39.154660 (+ 1320us) server_negotiation.cc:407] Connection header received
0807 08:39:39.155135 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:39.155142 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:39.155265 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:39.155332 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:39.155942 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43320: BlockingRecv error: recv got EOF from 127.0.0.1:43320 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 08:39:39.161737 10489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:39.160106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43324 (local address 127.9.174.61:42549)
0807 08:39:39.160219 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:39.160223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:39.160779 (+ 556us) server_negotiation.cc:407] Connection header received
0807 08:39:39.160966 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:39.160969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:39.161027 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:39.161079 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:39.161604 (+ 525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43324: BlockingRecv error: recv got EOF from 127.0.0.1:43324 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 08:39:39.168025 10489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:39.166256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43332 (local address 127.9.174.61:42549)
0807 08:39:39.166389 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:39.166393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:39.166974 (+ 581us) server_negotiation.cc:407] Connection header received
0807 08:39:39.167229 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:39.167233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:39.167292 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:39.167343 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:39.167889 (+ 546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43332: BlockingRecv error: recv got EOF from 127.0.0.1:43332 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 08:39:39.175737 10489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:39.174382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43346 (local address 127.9.174.61:42549)
0807 08:39:39.174519 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:39.174524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:39.174789 (+ 265us) server_negotiation.cc:407] Connection header received
0807 08:39:39.174926 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:39.174930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:39.175041 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:39.175141 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:39.175611 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43346: BlockingRecv error: recv got EOF from 127.0.0.1:43346 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 08:39:39.187016 10489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:39.185544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43352 (local address 127.9.174.61:42549)
0807 08:39:39.185683 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:39.185687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:39.186058 (+ 371us) server_negotiation.cc:407] Connection header received
0807 08:39:39.186231 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:39.186234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:39.186296 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:39.186389 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:39.186864 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43352: BlockingRecv error: recv got EOF from 127.0.0.1:43352 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 08:39:39.207018 10489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:39.205397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43358 (local address 127.9.174.61:42549)
0807 08:39:39.205572 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:39.205577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:39.205872 (+ 295us) server_negotiation.cc:407] Connection header received
0807 08:39:39.206085 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:39.206088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:39.206151 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:39.206248 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:39.206826 (+ 578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43358: BlockingRecv error: recv got EOF from 127.0.0.1:43358 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 08:39:39.243625 10489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:39.241832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43366 (local address 127.9.174.61:42549)
0807 08:39:39.242019 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:39.242024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:39.242348 (+ 324us) server_negotiation.cc:407] Connection header received
0807 08:39:39.242583 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:39.242586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:39.242655 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:39.242767 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:39.243448 (+ 681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43366: BlockingRecv error: recv got EOF from 127.0.0.1:43366 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260807 08:39:39.311844 10489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:39.310338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43372 (local address 127.9.174.61:42549)
0807 08:39:39.310501 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:39.310507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:39.310812 (+ 305us) server_negotiation.cc:407] Connection header received
0807 08:39:39.310964 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:39.310968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:39.311060 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:39.311173 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:39.311688 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43372: BlockingRecv error: recv got EOF from 127.0.0.1:43372 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 08:39:39.444003 10489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:39.442462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43378 (local address 127.9.174.61:42549)
0807 08:39:39.442670 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:39.442676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:39.442830 (+ 154us) server_negotiation.cc:407] Connection header received
0807 08:39:39.443066 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:39.443070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:39.443144 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:39.443254 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:39.443827 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43378: BlockingRecv error: recv got EOF from 127.0.0.1:43378 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 08:39:39.703950 10489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:39.702609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43380 (local address 127.9.174.61:42549)
0807 08:39:39.702812 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:39.702817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:39.702835 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:39:39.702985 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:39.702988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:39.703102 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:39.703182 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:39.703757 (+ 575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43380: BlockingRecv error: recv got EOF from 127.0.0.1:43380 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 08:39:39.911397 10604 catalog_manager.cc:1259] Loaded cluster ID: 9dbb78ad3fdd4b248ee86ca0d1138f01
I20260807 08:39:39.911506 10604 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9: loading cluster ID for follower catalog manager: success
I20260807 08:39:39.913579 10604 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9: acquiring CA information for follower catalog manager: success
I20260807 08:39:39.914947 10604 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d7438d7f75e0454cb234375d374403a9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:39:40.100116 10672 catalog_manager.cc:1259] Loaded cluster ID: 9dbb78ad3fdd4b248ee86ca0d1138f01
I20260807 08:39:40.100275 10672 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269: loading cluster ID for follower catalog manager: success
I20260807 08:39:40.102555 10672 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269: acquiring CA information for follower catalog manager: success
I20260807 08:39:40.104511 10672 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9ecf3cc33aee4b099436fc482a254269: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:39:40.223757 10776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:40.219055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43384 (local address 127.9.174.61:42549)
0807 08:39:40.219430 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:40.219435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:40.219454 (+ 19us) server_negotiation.cc:407] Connection header received
0807 08:39:40.219534 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:40.219537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:40.219714 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:40.219848 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:39:40.220186 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:40.220823 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:40.221781 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:40.222093 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:40.222204 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:40.222553 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:39:40.223300 (+ 747us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:39:40.223442 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:40.223503 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 08:39:40.223574 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":112,"threads_started":1}
I20260807 08:39:40.232569 10782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:40.227998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43848 (local address 127.9.174.60:45665)
0807 08:39:40.228296 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:40.228300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:40.228315 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:39:40.228432 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:40.228434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:40.228611 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:40.228735 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:39:40.228963 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:40.229562 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:40.230611 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:40.230877 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:40.231103 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:40.231448 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:39:40.232055 (+ 607us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:39:40.232167 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:40.232247 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 08:39:40.232312 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":107,"threads_started":1}
I20260807 08:39:40.241245 10776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:40.236682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43398 (local address 127.9.174.61:42549)
0807 08:39:40.236924 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:40.236932 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:40.237208 (+ 276us) server_negotiation.cc:407] Connection header received
0807 08:39:40.237334 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:40.237336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:40.237401 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:40.237463 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:39:40.237740 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:40.238529 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:40.239838 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:40.240184 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:40.240300 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:40.240570 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:39:40.240671 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:39:40.240744 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:40.240966 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 08:39:40.241035 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 08:39:40.241997 10794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:40.236639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53912 (local address 127.9.174.62:38725)
0807 08:39:40.237421 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:40.237425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:40.237438 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:39:40.237585 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:40.237588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:40.237821 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:40.238003 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:39:40.238302 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:40.239044 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:40.240103 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:40.240438 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:40.240545 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:40.241013 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:39:40.241494 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:39:40.241591 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:40.241718 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 08:39:40.241787 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":137,"threads_started":1}
I20260807 08:39:40.242211 10782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:40.237103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43864 (local address 127.9.174.60:45665)
0807 08:39:40.237231 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:40.237236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:40.238373 (+ 1137us) server_negotiation.cc:407] Connection header received
0807 08:39:40.238520 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:40.238523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:40.238580 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:40.238664 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:39:40.239057 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:40.239607 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:40.240757 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:40.241100 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:40.241199 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:40.241502 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:39:40.241597 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:39:40.241667 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:40.242021 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 08:39:40.242082 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 08:39:40.246009 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 10276
I20260807 08:39:40.273656 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 10386
I20260807 08:39:40.287554 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 10490
2026-08-07T08:39:40Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3641 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:40 dist-test-slave-h3ft krb5kdc[10800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:40 dist-test-slave-h3ft krb5kdc[10800](info): set up 2 sockets
Aug 07 08:39:40 dist-test-slave-h3ft krb5kdc[10800](info): commencing operation
krb5kdc: starting...
W20260807 08:39:41.631597 9912 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/test-user.keytab.
Aug 07 08:39:41 dist-test-slave-h3ft krb5kdc[10800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.174.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
Entry for principal kudu/127.9.174.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab.
I20260807 08:39:41.784754 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:36265
--webserver_interface=127.9.174.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:44747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:36265,127.9.174.61:40989,127.9.174.60:46185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:41.931666 10819 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:39:41.932093 10819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:41.932175 10819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:41.932276 10819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:41.936501 10819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:41.936617 10819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:41.936657 10819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:41.936697 10819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:41.936733 10819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:41.942941 10819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:44747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:36265,127.9.174.61:40989,127.9.174.60:46185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.62.keytab
--principal=kudu/127.9.174.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.62:36265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.174.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:41.944507 10819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:41.945868 10819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:41.956700 10828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:41.956700 10832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:41.956727 10829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:41.957118 10819 server_base.cc:1034] running on GCE node
I20260807 08:39:41.958019 10819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:41.959659 10819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:41.960877 10819 hybrid_clock.cc:648] HybridClock initialized: now 1786091981960829 us; error 65 us; skew 500 ppm
Aug 07 08:39:41 dist-test-slave-h3ft krb5kdc[10800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091981, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:41.969571 10819 init.cc:377] Logged in from keytab as kudu/127.9.174.62@KRBTEST.COM (short username kudu)
I20260807 08:39:41.970902 10819 webserver.cc:460] Webserver started at http://127.9.174.62:40691/ using document root <none> and password file <none>
I20260807 08:39:41.971683 10819 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:41.971791 10819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:41.972107 10819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:41.974752 10819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/data/instance:
uuid: "54229f45723f46deadbd15943666dae5"
format_stamp: "Formatted at 2026-08-07 08:39:41 on dist-test-slave-h3ft"
server_key: "7e5f274fcaa2f300eefc445d0e4c4765"
server_key_iv: "c9571e9833884ca272f7fdefcc4eff3f"
server_key_version: "encryptionkey@0"
I20260807 08:39:41.975540 10819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/wal/instance:
uuid: "54229f45723f46deadbd15943666dae5"
format_stamp: "Formatted at 2026-08-07 08:39:41 on dist-test-slave-h3ft"
server_key: "7e5f274fcaa2f300eefc445d0e4c4765"
server_key_iv: "c9571e9833884ca272f7fdefcc4eff3f"
server_key_version: "encryptionkey@0"
I20260807 08:39:41.980168 10819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:39:41.983363 10842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:41.985112 10819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 08:39:41.985309 10819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/wal
uuid: "54229f45723f46deadbd15943666dae5"
format_stamp: "Formatted at 2026-08-07 08:39:41 on dist-test-slave-h3ft"
server_key: "7e5f274fcaa2f300eefc445d0e4c4765"
server_key_iv: "c9571e9833884ca272f7fdefcc4eff3f"
server_key_version: "encryptionkey@0"
I20260807 08:39:41.985473 10819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:42.009234 10819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:42.013198 10819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:42.013473 10819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:42.022600 10819 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.62:36265
I20260807 08:39:42.022964 10913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.62:36265 every 8 connection(s)
I20260807 08:39:42.023928 10819 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:42.025651 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 10819
I20260807 08:39:42.025811 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-0/wal/instance
I20260807 08:39:42.026180 9912 external_mini_cluster.cc:1428] Setting key 54750d65e088d92ac4d66e7724666d4f
I20260807 08:39:42.027724 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:42.034411 10916 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.174.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.61@KRBTEST.COM" created.
Aug 07 08:39:42 dist-test-slave-h3ft krb5kdc[10800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091981, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:39:42.053179 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.037920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:36265 (local address 127.0.0.1:38508)
0807 08:39:42.038763 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.038807 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:42.039881 (+ 1074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:42.040689 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:42.040695 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:42.041177 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:42.041538 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.041555 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.042835 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.042838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.043831 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.043844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.044578 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.044585 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.044734 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.045469 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:42.045497 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:42.047455 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:42.049605 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.049616 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.049627 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.049879 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.050231 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.050236 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.050238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.050357 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.051106 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:42.051114 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:42.051270 (+ 156us) client_negotiation.cc:769] Sending connection context
0807 08:39:42.051460 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 08:39:42.051956 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":109,"thread_start_us":184,"threads_started":1}
I20260807 08:39:42.053179 10920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.037920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38508 (local address 127.9.174.62:36265)
0807 08:39:42.038451 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.038472 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.038940 (+ 468us) server_negotiation.cc:407] Connection header received
0807 08:39:42.040205 (+ 1265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.040210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.040438 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.040689 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.041695 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.042679 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.043983 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.044387 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.044535 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.047640 (+ 3105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:42.047646 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:42.047657 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:42.047679 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:42.049471 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.050014 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.050019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.050021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.050130 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.050460 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.050469 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.050471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.050921 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:42.051108 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:42.051539 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 08:39:42.051829 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"spinlock_wait_cycles":43392,"thread_start_us":182,"threads_started":1}
I20260807 08:39:42.056308 10916 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:39:42.057296 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.056946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:40989 (local address 127.0.0.1:54890)
0807 08:39:42.057110 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.057213 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 08:39:42.057570 10847 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.174.61:40989: connect: Connection refused (error 111)
I20260807 08:39:42.058660 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.058440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:40989 (local address 127.0.0.1:54906)
0807 08:39:42.058552 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.058598 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 08:39:42.058862 10916 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.61:40989: Network error: Client connection negotiation failed: client connection to 127.9.174.61:40989: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.61.keytab.
Entry for principal kudu/127.9.174.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.61.keytab.
I20260807 08:39:42.062362 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.61:40989
--webserver_interface=127.9.174.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.61.keytab
--principal=kudu/127.9.174.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:44747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:36265,127.9.174.61:40989,127.9.174.60:46185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:39:42.108309 10916 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } attempt: 1
I20260807 08:39:42.109531 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.109134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:40989 (local address 127.0.0.1:54914)
0807 08:39:42.109331 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.109425 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 08:39:42.110554 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.110268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:40989 (local address 127.0.0.1:54930)
0807 08:39:42.110427 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.110483 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260807 08:39:42.110733 10916 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.61:40989: Network error: Client connection negotiation failed: client connection to 127.9.174.61:40989: connect: Connection refused (error 111)
I20260807 08:39:42.178974 10916 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } attempt: 2
I20260807 08:39:42.180335 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.179854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:40989 (local address 127.0.0.1:54936)
0807 08:39:42.180130 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.180216 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260807 08:39:42.181221 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.180960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:40989 (local address 127.0.0.1:54942)
0807 08:39:42.181092 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.181144 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 08:39:42.181416 10916 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.61:40989: Network error: Client connection negotiation failed: client connection to 127.9.174.61:40989: connect: Connection refused (error 111)
W20260807 08:39:42.200202 10924 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:39:42.200574 10924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:42.200655 10924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:42.200729 10924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:42.205034 10924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:42.205098 10924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:42.205120 10924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:42.205137 10924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:42.205193 10924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:42.210956 10924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:44747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:36265,127.9.174.61:40989,127.9.174.60:46185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.61.keytab
--principal=kudu/127.9.174.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.61:40989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.174.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:42.212399 10924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:42.213703 10924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:42.224368 10933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:42.224390 10940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:42.224443 10936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:42.224455 10924 server_base.cc:1034] running on GCE node
I20260807 08:39:42.225417 10924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:42.226794 10924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:42.227977 10924 hybrid_clock.cc:648] HybridClock initialized: now 1786091982227965 us; error 33 us; skew 500 ppm
Aug 07 08:39:42 dist-test-slave-h3ft krb5kdc[10800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091982, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:42.231359 10924 init.cc:377] Logged in from keytab as kudu/127.9.174.61@KRBTEST.COM (short username kudu)
I20260807 08:39:42.232443 10924 webserver.cc:460] Webserver started at http://127.9.174.61:42045/ using document root <none> and password file <none>
I20260807 08:39:42.233139 10924 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:42.233204 10924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:42.233498 10924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:42.236059 10924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/data/instance:
uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb"
format_stamp: "Formatted at 2026-08-07 08:39:42 on dist-test-slave-h3ft"
server_key: "14118512e30a19c76e7c7edda0eec173"
server_key_iv: "876774a529d265115758ebc470aaac6f"
server_key_version: "encryptionkey@0"
I20260807 08:39:42.236701 10924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/wal/instance:
uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb"
format_stamp: "Formatted at 2026-08-07 08:39:42 on dist-test-slave-h3ft"
server_key: "14118512e30a19c76e7c7edda0eec173"
server_key_iv: "876774a529d265115758ebc470aaac6f"
server_key_version: "encryptionkey@0"
I20260807 08:39:42.241046 10924 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 08:39:42.244035 10951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:42.245579 10924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 08:39:42.245752 10924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/wal
uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb"
format_stamp: "Formatted at 2026-08-07 08:39:42 on dist-test-slave-h3ft"
server_key: "14118512e30a19c76e7c7edda0eec173"
server_key_iv: "876774a529d265115758ebc470aaac6f"
server_key_version: "encryptionkey@0"
I20260807 08:39:42.245883 10924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:42.261502 10924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:42.265744 10924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:42.266006 10924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:42.272658 10916 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } attempt: 3
I20260807 08:39:42.273957 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.273455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:40989 (local address 127.0.0.1:54958)
0807 08:39:42.273662 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.273744 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 08:39:42.274938 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.274611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:40989 (local address 127.0.0.1:54972)
0807 08:39:42.274755 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.274830 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.61:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 08:39:42.275120 10916 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.61:40989: Network error: Client connection negotiation failed: client connection to 127.9.174.61:40989: connect: Connection refused (error 111)
I20260807 08:39:42.275154 10924 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.61:40989
I20260807 08:39:42.275532 11022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.61:40989 every 8 connection(s)
I20260807 08:39:42.276398 10924 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/data/info.pb
I20260807 08:39:42.280025 11024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:42.282209 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 10924
I20260807 08:39:42.282353 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-1/wal/instance
I20260807 08:39:42.282724 9912 external_mini_cluster.cc:1428] Setting key 3e3baf38c92033ed445654f78ac4eb59
I20260807 08:39:42.286201 11024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.174.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.174.60@KRBTEST.COM" created.
Aug 07 08:39:42 dist-test-slave-h3ft krb5kdc[10800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091982, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.61@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:39:42.303506 10920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.288831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38516 (local address 127.9.174.62:36265)
0807 08:39:42.289016 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.289020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.290414 (+ 1394us) server_negotiation.cc:407] Connection header received
0807 08:39:42.291681 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.291685 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.291772 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.291876 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.293272 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.293859 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.295256 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.295547 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.295668 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.299131 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:42.299135 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:42.299137 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:42.299170 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:42.300949 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.301627 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.301636 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.301637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.301724 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.302115 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.302118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.302120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.302359 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:42.302474 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:42.303127 (+ 653us) server_negotiation.cc:299] Negotiation successful
0807 08:39:42.303299 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 08:39:42.304620 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.289420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:36265 (local address 127.0.0.1:38516)
0807 08:39:42.290230 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.290283 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:42.291433 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:42.292085 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:42.292092 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:42.292509 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:42.293106 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.293120 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.294017 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.294020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.295114 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.295124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.295737 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.295741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.295992 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.296753 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:42.296772 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:42.298846 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:42.301137 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.301144 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.301183 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.301505 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.301831 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.301835 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.301837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.302003 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.302473 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:42.302482 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:42.302811 (+ 329us) client_negotiation.cc:769] Sending connection context
0807 08:39:42.303046 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 08:39:42.303501 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":236,"threads_started":1}
I20260807 08:39:42.306550 11024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:39:42 dist-test-slave-h3ft krb5kdc[10800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091982, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.61@KRBTEST.COM for kudu/127.9.174.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.174.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.60.keytab.
Entry for principal kudu/127.9.174.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.60.keytab.
I20260807 08:39:42.318005 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.307504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54978 (local address 127.9.174.61:40989)
0807 08:39:42.308001 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.308013 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.308036 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:39:42.308076 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.308081 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.308268 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.308407 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.308936 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.309845 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.310741 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.311379 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.311473 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.313788 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:42.313799 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:42.313813 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:42.313837 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:42.315708 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.316347 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.316353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.316354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.316448 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.316773 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.316776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.316777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.317083 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:42.317204 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:42.317676 (+ 472us) server_negotiation.cc:299] Negotiation successful
0807 08:39:42.317824 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":266,"threads_started":1}
I20260807 08:39:42.318017 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.307348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:40989 (local address 127.0.0.1:54978)
0807 08:39:42.307555 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.307596 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:42.307707 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:42.308425 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:42.308430 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:42.308643 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:42.308752 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.308758 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.309977 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.309981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.310622 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.310632 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.311512 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.311515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.311593 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.312217 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:42.312230 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:42.313645 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:42.315815 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.315818 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.315820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.316249 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.316559 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.316561 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.316562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.316682 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.317278 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:42.317282 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:42.317398 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 08:39:42.317529 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 08:39:42.317718 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 08:39:42.318789 9912 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1t_l5C/build/debug/bin/kudu
/tmp/dist-test-task1t_l5C/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.60:46185
--webserver_interface=127.9.174.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.60.keytab
--principal=kudu/127.9.174.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.174.20:44747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.174.62:36265,127.9.174.61:40989,127.9.174.60:46185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:39:42.320686 11024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:39:42.321611 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.321264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:46185 (local address 127.0.0.1:60740)
0807 08:39:42.321399 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.321509 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:46185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 08:39:42.321821 10954 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.174.60:46185: connect: Connection refused (error 111)
I20260807 08:39:42.322875 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.322604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:46185 (local address 127.0.0.1:60744)
0807 08:39:42.322719 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.322785 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:46185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 08:39:42.323077 11024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:46185: Network error: Client connection negotiation failed: client connection to 127.9.174.60:46185: connect: Connection refused (error 111)
I20260807 08:39:42.372422 11024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } attempt: 1
I20260807 08:39:42.373723 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.373277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:46185 (local address 127.0.0.1:60752)
0807 08:39:42.373499 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.373590 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:46185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 08:39:42.374564 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.374316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:46185 (local address 127.0.0.1:60754)
0807 08:39:42.374435 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.374489 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:46185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 08:39:42.374751 11024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:46185: Network error: Client connection negotiation failed: client connection to 127.9.174.60:46185: connect: Connection refused (error 111)
I20260807 08:39:42.418346 10916 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } attempt: 4
Aug 07 08:39:42 dist-test-slave-h3ft krb5kdc[10800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091981, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for kudu/127.9.174.61@KRBTEST.COM
I20260807 08:39:42.429574 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.419363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54984 (local address 127.9.174.61:40989)
0807 08:39:42.419604 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.419611 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.419721 (+ 110us) server_negotiation.cc:407] Connection header received
0807 08:39:42.419931 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.419935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.420000 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.420104 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.420673 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.421678 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.422968 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.423297 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.423420 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.426078 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:42.426082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:42.426084 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:42.426106 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:42.427760 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.428231 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.428239 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.428241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.428319 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.428619 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.428623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.428625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.428830 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:42.428944 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:42.429253 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 08:39:42.429383 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260807 08:39:42.429574 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.419293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:40989 (local address 127.0.0.1:54984)
0807 08:39:42.419599 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.419620 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:42.419807 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:42.420145 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:42.420148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:42.420387 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:42.420529 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.420536 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.421819 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.421823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.422826 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.422837 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.423439 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.423442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.423581 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.424232 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:42.424246 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:42.425881 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:42.427886 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.427890 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.427892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.428118 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.428419 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.428422 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.428424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.428520 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.428933 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:42.428936 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:42.429079 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 08:39:42.429198 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 08:39:42.429364 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200}
I20260807 08:39:42.430653 10916 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:39:42.431521 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.431257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:46185 (local address 127.0.0.1:60766)
0807 08:39:42.431381 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.431439 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:46185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 08:39:42.432314 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.432041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:46185 (local address 127.0.0.1:60772)
0807 08:39:42.432175 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.432243 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:46185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 08:39:42.432539 10916 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:46185: Network error: Client connection negotiation failed: client connection to 127.9.174.60:46185: connect: Connection refused (error 111)
I20260807 08:39:42.442945 11024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } attempt: 2
I20260807 08:39:42.443884 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.443529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:46185 (local address 127.0.0.1:60782)
0807 08:39:42.443707 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.443781 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:46185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 08:39:42.444792 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.444526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:46185 (local address 127.0.0.1:60798)
0807 08:39:42.444646 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.444709 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:46185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 08:39:42.444984 11024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:46185: Network error: Client connection negotiation failed: client connection to 127.9.174.60:46185: connect: Connection refused (error 111)
W20260807 08:39:42.458350 11032 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:39:42.458701 11032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:42.458750 11032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:42.458860 11032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:42.462984 11032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:42.463099 11032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:42.463127 11032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:42.463157 11032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:42.463189 11032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:42.468779 11032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.174.20:44747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.174.62:36265,127.9.174.61:40989,127.9.174.60:46185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/krb5kdc/kudu_127.9.174.60.keytab
--principal=kudu/127.9.174.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.174.60:46185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.174.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:39:42.470337 11032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:42.471647 11032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:42.481477 11043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:42.481451 11046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:42.481460 11042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:42.481791 11032 server_base.cc:1034] running on GCE node
I20260807 08:39:42.482676 11032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:42.484103 11032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:42.485273 11032 hybrid_clock.cc:648] HybridClock initialized: now 1786091982485259 us; error 30 us; skew 500 ppm
Aug 07 08:39:42 dist-test-slave-h3ft krb5kdc[10800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091982, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:42.488600 11032 init.cc:377] Logged in from keytab as kudu/127.9.174.60@KRBTEST.COM (short username kudu)
I20260807 08:39:42.489796 11032 webserver.cc:460] Webserver started at http://127.9.174.60:39899/ using document root <none> and password file <none>
I20260807 08:39:42.490507 11032 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:42.490603 11032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:42.490890 11032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:42.491757 10916 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } attempt: 1
I20260807 08:39:42.492714 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.492358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:46185 (local address 127.0.0.1:60812)
0807 08:39:42.492534 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.492607 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:46185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 08:39:42.493594 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.493336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:46185 (local address 127.0.0.1:60814)
0807 08:39:42.493474 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.493527 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.174.60:46185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 08:39:42.493824 10916 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.174.60:46185: Network error: Client connection negotiation failed: client connection to 127.9.174.60:46185: connect: Connection refused (error 111)
I20260807 08:39:42.493861 11032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/data/instance:
uuid: "7f1cfce791594fa786978723b085ec0c"
format_stamp: "Formatted at 2026-08-07 08:39:42 on dist-test-slave-h3ft"
server_key: "cbcb8d0c02183c39f69384bc7cc5be90"
server_key_iv: "296411fbba8f26653dd7174ee74e9939"
server_key_version: "encryptionkey@0"
I20260807 08:39:42.494572 11032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/wal/instance:
uuid: "7f1cfce791594fa786978723b085ec0c"
format_stamp: "Formatted at 2026-08-07 08:39:42 on dist-test-slave-h3ft"
server_key: "cbcb8d0c02183c39f69384bc7cc5be90"
server_key_iv: "296411fbba8f26653dd7174ee74e9939"
server_key_version: "encryptionkey@0"
I20260807 08:39:42.498895 11032 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260807 08:39:42.501839 11056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:42.503458 11032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:39:42.503597 11032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/data,/tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/wal
uuid: "7f1cfce791594fa786978723b085ec0c"
format_stamp: "Formatted at 2026-08-07 08:39:42 on dist-test-slave-h3ft"
server_key: "cbcb8d0c02183c39f69384bc7cc5be90"
server_key_iv: "296411fbba8f26653dd7174ee74e9939"
server_key_version: "encryptionkey@0"
I20260807 08:39:42.503719 11032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:42.515426 11032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:42.519104 11032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:42.519336 11032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:42.528184 11032 rpc_server.cc:307] RPC server started. Bound to: 127.9.174.60:46185
I20260807 08:39:42.528545 11125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.174.60:46185 every 8 connection(s)
I20260807 08:39:42.529469 11032 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/data/info.pb
I20260807 08:39:42.533174 11127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:42.536201 11024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } attempt: 3
I20260807 08:39:42.538076 11127 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:39:42.539346 9912 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1t_l5C/build/debug/bin/kudu as pid 11032
I20260807 08:39:42.539513 9912 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1t_l5C/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091655732880-9912-0/minicluster-data/master-2/wal/instance
I20260807 08:39:42.539871 9912 external_mini_cluster.cc:1428] Setting key e1e1a72628321613dcb9ae9656ef94ba
Aug 07 08:39:42 dist-test-slave-h3ft krb5kdc[10800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091982, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.61@KRBTEST.COM for kudu/127.9.174.60@KRBTEST.COM
Aug 07 08:39:42 dist-test-slave-h3ft krb5kdc[10800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091982, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.60@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
Aug 07 08:39:42 dist-test-slave-h3ft krb5kdc[10800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.60@KRBTEST.COM
I20260807 08:39:42.553148 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.537007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:46185 (local address 127.0.0.1:60818)
0807 08:39:42.537228 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.537250 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:42.537383 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:42.540399 (+ 3016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:42.540404 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:42.540668 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:42.540823 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.540842 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.543109 (+ 2267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.543115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.543692 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.543704 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.544753 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.544757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.544861 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.545530 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:42.545544 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:42.547385 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:42.550075 (+ 2690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.550079 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.550082 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.550402 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.551499 (+ 1097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.551502 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.551504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.551624 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.552455 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:42.552461 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:42.552615 (+ 154us) client_negotiation.cc:769] Sending connection context
0807 08:39:42.552741 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 08:39:42.552918 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 08:39:42.553781 10920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.541545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38532 (local address 127.9.174.62:36265)
0807 08:39:42.541703 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.541707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.541724 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:39:42.541784 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.541789 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.541911 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.542063 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.543154 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.543666 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.544646 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.544979 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.545107 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.549153 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:42.549157 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:42.549160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:42.549201 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:42.551126 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.551785 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.551790 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.551791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.551898 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.552231 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.552236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.552237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.552533 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:42.552651 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:42.553445 (+ 794us) server_negotiation.cc:299] Negotiation successful
0807 08:39:42.553594 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 08:39:42.554529 11132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.540556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.62:36265 (local address 127.0.0.1:38532)
0807 08:39:42.541200 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.541258 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:42.541376 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:42.542252 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:42.542257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:42.542634 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:42.542972 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.542980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.543801 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.543805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.544510 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.544521 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.545557 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.545561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.545671 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.547100 (+ 1429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:42.547117 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:42.548918 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:42.551234 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.551246 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.551260 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.551626 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.552020 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.552023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.552025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.552118 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.552662 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:42.552669 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:42.552831 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 08:39:42.552984 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 08:39:42.554096 (+ 1112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":920,"spinlock_wait_cycles":12288,"thread_start_us":157,"threads_started":1}
I20260807 08:39:42.554530 11129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.537811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60818 (local address 127.9.174.60:46185)
0807 08:39:42.538379 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.538401 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.538452 (+ 51us) server_negotiation.cc:407] Connection header received
0807 08:39:42.539493 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.539521 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.540015 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.540430 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.541042 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.542931 (+ 1889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.543849 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.544603 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.544776 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.547564 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:42.547574 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:42.547591 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:42.547622 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:42.549964 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.551253 (+ 1289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.551258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.551260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.551383 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.551749 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.551753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.551756 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.552333 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:42.552488 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:42.552894 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 08:39:42.553235 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":397,"spinlock_wait_cycles":7040,"thread_start_us":186,"threads_started":1}
I20260807 08:39:42.554765 11135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.541061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60828 (local address 127.9.174.60:46185)
0807 08:39:42.541588 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.541597 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.541685 (+ 88us) server_negotiation.cc:407] Connection header received
0807 08:39:42.541833 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.541837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.541886 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.541974 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.543822 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.544603 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.545843 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.546272 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.546355 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.549553 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:42.549557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:42.549559 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:42.549589 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:42.552499 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.553036 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.553039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.553041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.553127 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.553430 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.553433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.553434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.553626 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:42.553793 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:42.554458 (+ 665us) server_negotiation.cc:299] Negotiation successful
0807 08:39:42.554579 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":102,"server-negotiator.queue_time_us":464,"spinlock_wait_cycles":1280,"thread_start_us":178,"threads_started":1}
I20260807 08:39:42.556716 11127 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:39:42.560091 11024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb: Bootstrap starting.
I20260807 08:39:42.561000 10916 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } attempt: 2
Aug 07 08:39:42 dist-test-slave-h3ft krb5kdc[10800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091982, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.60@KRBTEST.COM for kudu/127.9.174.61@KRBTEST.COM
I20260807 08:39:42.563633 11024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb: Neither blocks nor log segments found. Creating new log.
Aug 07 08:39:42 dist-test-slave-h3ft krb5kdc[10800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.62@KRBTEST.COM
I20260807 08:39:42.564711 11024 log.cc:824] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:39:42 dist-test-slave-h3ft krb5kdc[10800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091981, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.62@KRBTEST.COM for kudu/127.9.174.60@KRBTEST.COM
I20260807 08:39:42.568583 11024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb: No bootstrap required, opened a new log
I20260807 08:39:42.568688 11132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.557292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.61:40989 (local address 127.0.0.1:54990)
0807 08:39:42.557712 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.557728 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:42.557843 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:42.558193 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:42.558199 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:42.558363 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:42.558454 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.558461 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.559420 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.559425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.560053 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.560064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.560609 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.560613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.560721 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.561851 (+ 1130us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:42.561876 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:42.564086 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:42.566772 (+ 2686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.566775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.566777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.567093 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.567434 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.567437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.567439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.567533 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.568115 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:42.568118 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:42.568213 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 08:39:42.568330 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 08:39:42.568490 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344}
I20260807 08:39:42.569871 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.557392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54990 (local address 127.9.174.61:40989)
0807 08:39:42.557527 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.557530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.557785 (+ 255us) server_negotiation.cc:407] Connection header received
0807 08:39:42.557936 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.557940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.557999 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.558054 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.558567 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.559310 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.560182 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.560481 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.560591 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.564242 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:42.564247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:42.564249 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:42.564276 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:42.566643 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.567216 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.567220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.567222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.567314 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.567643 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.567646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.567647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.567935 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:42.568035 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:42.568447 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 08:39:42.569720 (+ 1273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:39:42.569883 10920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.557936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38544 (local address 127.9.174.62:36265)
0807 08:39:42.558044 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.558048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.558126 (+ 78us) server_negotiation.cc:407] Connection header received
0807 08:39:42.558257 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.558260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.558306 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.558395 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.559130 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.559818 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.560733 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.561092 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.561167 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.564954 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:42.564958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:42.564960 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:42.564983 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:42.567294 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.567835 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.567843 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.567845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.567984 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.568345 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.568349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.568351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.568569 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:42.569458 (+ 889us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:42.569536 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 08:39:42.569708 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:39:42.571354 11129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.561478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60844 (local address 127.9.174.60:46185)
0807 08:39:42.561846 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.561850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.561870 (+ 20us) server_negotiation.cc:407] Connection header received
0807 08:39:42.561904 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.561906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.561948 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.562494 (+ 546us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.562576 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.563283 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.564343 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.564684 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.564797 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.567482 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:42.567488 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:42.567493 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:42.567521 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:42.569164 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.570020 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.570023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.570025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.570109 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.570418 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.570421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.570423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.570647 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:42.570750 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:42.571099 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 08:39:42.571208 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:39:42.571353 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.561412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:46185 (local address 127.0.0.1:60844)
0807 08:39:42.561620 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.561633 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:42.561705 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:42.562092 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:42.562095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:42.562286 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:42.562381 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.562387 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.563413 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.563417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.564247 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.564257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.564802 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.564807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.564922 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.565508 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:42.565530 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:42.567318 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:42.569557 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.569561 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.569563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.569887 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.570213 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.570215 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.570218 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.570309 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.570799 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:42.570802 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:42.570910 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 08:39:42.571046 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 08:39:42.571196 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260807 08:39:42.571955 11127 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:39:42.573575 11024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } }
I20260807 08:39:42.573844 11024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b8749f0d6fb49afbdf3cfd3a7e8a7fb, State: Initialized, Role: FOLLOWER
I20260807 08:39:42.574225 11024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } }
I20260807 08:39:42.574826 11137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } } }
I20260807 08:39:42.575099 11137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:39:42.575434 10916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5: Bootstrap starting.
I20260807 08:39:42.575980 11024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 08:39:42 dist-test-slave-h3ft krb5kdc[10800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.174.61@KRBTEST.COM
I20260807 08:39:42.579562 10916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:42.580662 10916 log.cc:824] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:42.582711 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.571598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54992 (local address 127.9.174.61:40989)
0807 08:39:42.572672 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.572677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.572689 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:39:42.572726 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.572729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.572769 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.572847 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.573607 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.574344 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.575445 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.575735 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.575842 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.578895 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:42.578901 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:42.578908 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:42.578939 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:42.580663 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.581227 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.581230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.581232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.581322 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.581686 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.581690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.581692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.581901 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:42.582056 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:42.582442 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 08:39:42.582565 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":985}
I20260807 08:39:42.583218 10916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5: No bootstrap required, opened a new log
Aug 07 08:39:42 dist-test-slave-h3ft krb5kdc[10800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091982, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.174.60@KRBTEST.COM for kudu/127.9.174.60@KRBTEST.COM
I20260807 08:39:42.580945 10924 master_runner.cc:186] Error getting master registration for 127.9.174.62:36265: OK, instance_id { permanent_uuid: "54229f45723f46deadbd15943666dae5" instance_seqno: 1786091982020011 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:39:42.586110 10924 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:39:42.587231 10916 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } }
I20260807 08:39:42.587476 10916 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54229f45723f46deadbd15943666dae5, State: Initialized, Role: FOLLOWER
I20260807 08:39:42.587858 10916 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } }
I20260807 08:39:42.588201 11129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.572676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60850 (local address 127.9.174.60:46185)
0807 08:39:42.578737 (+ 6061us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.578742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.578757 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:39:42.578795 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.578801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.578875 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.578956 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.579515 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.580233 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.581214 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.581541 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.581623 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.584144 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:42.584147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:42.584150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:42.584170 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:42.585819 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.586309 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.586314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.586315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.586404 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.586722 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.586727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.586728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.586982 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:42.587384 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:42.587829 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 08:39:42.587981 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":5977}
I20260807 08:39:42.588203 11132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.572463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.174.60:46185 (local address 127.0.0.1:60850)
0807 08:39:42.573001 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:42.573015 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:42.573084 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:42.579032 (+ 5948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:42.579035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:42.579235 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:42.579354 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.579365 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.580345 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.580349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.581087 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:42.581098 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.581729 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.581732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:42.581809 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.582285 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:42.582300 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:42.584006 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:42.585939 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.585942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.585944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.586196 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.586516 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:42.586518 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:42.586520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:42.586612 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:42.587394 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:42.587398 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:42.587637 (+ 239us) client_negotiation.cc:769] Sending connection context
0807 08:39:42.587758 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 08:39:42.587904 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455}
I20260807 08:39:42.588738 11162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } } }
I20260807 08:39:42.588864 11162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:39:42.589440 10916 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:42.591601 11127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c: Bootstrap starting.
I20260807 08:39:42.593942 9912 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:42.595285 10877 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "54229f45723f46deadbd15943666dae5"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38544
W20260807 08:39:42.595407 11158 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:39:42.595407 11178 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:39:42.595479 11178 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260807 08:39:42.595479 11158 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:39:42.595536 10877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 08:39:42.595655 10877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:42.597940 10877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } }
I20260807 08:39:42.598593 11127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:42.599630 10877 leader_election.cc:288] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [CANDIDATE]: Term 1 election: Requested vote from peers 3b8749f0d6fb49afbdf3cfd3a7e8a7fb (127.9.174.61:40989), 7f1cfce791594fa786978723b085ec0c (127.9.174.60:46185)
I20260807 08:39:42.600049 11127 log.cc:824] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:42.602231 11127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c: No bootstrap required, opened a new log
I20260807 08:39:42.598888 10819 master_runner.cc:186] Error getting master registration for 127.9.174.60:46185: OK, instance_id { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" instance_seqno: 1786091982525745 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:39:42.602703 10819 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:39:42.603430 10987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54229f45723f46deadbd15943666dae5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb"
I20260807 08:39:42.603718 10987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:42.606034 10987 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54229f45723f46deadbd15943666dae5 in term 1.
I20260807 08:39:42.606684 10847 leader_election.cc:302] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b8749f0d6fb49afbdf3cfd3a7e8a7fb, 54229f45723f46deadbd15943666dae5; no voters:
I20260807 08:39:42.607170 11162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:42.607133 11127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } }
I20260807 08:39:42.607451 11162 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [term 1 LEADER]: Becoming Leader. State: Replica: 54229f45723f46deadbd15943666dae5, State: Running, Role: LEADER
I20260807 08:39:42.607434 11127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f1cfce791594fa786978723b085ec0c, State: Initialized, Role: FOLLOWER
I20260807 08:39:42.607836 11162 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } }
I20260807 08:39:42.607915 11127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } }
I20260807 08:39:42.609738 11127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:42.609793 11093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54229f45723f46deadbd15943666dae5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7f1cfce791594fa786978723b085ec0c"
I20260807 08:39:42.610031 11093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:42.610037 11162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54229f45723f46deadbd15943666dae5. Latest consensus state: current_term: 1 leader_uuid: "54229f45723f46deadbd15943666dae5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } } }
I20260807 08:39:42.610180 11162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:42.610842 11195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:42.612097 11195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:42.612604 11093 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54229f45723f46deadbd15943666dae5 in term 1.
I20260807 08:39:42.618770 11192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } } }
I20260807 08:39:42.618979 11192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:39:42.620010 11093 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c [term 1 FOLLOWER]: Refusing update from remote peer 54229f45723f46deadbd15943666dae5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:39:42.620752 11186 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:39:42.620967 10987 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb [term 1 FOLLOWER]: Refusing update from remote peer 54229f45723f46deadbd15943666dae5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:39:42.621719 11186 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 08:39:42.621768 11209 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:39:42.621850 11209 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:39:42.626422 11192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54229f45723f46deadbd15943666dae5. Latest consensus state: current_term: 1 leader_uuid: "54229f45723f46deadbd15943666dae5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } } }
I20260807 08:39:42.626564 11192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:39:42.627581 11137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54229f45723f46deadbd15943666dae5. Latest consensus state: current_term: 1 leader_uuid: "54229f45723f46deadbd15943666dae5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } } }
I20260807 08:39:42.627741 11137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:39:42.629885 11211 mvcc.cc:205] Tried to move back new op lower bound from 7315832760802013184 to 7315832760765968384. Current Snapshot: MvccSnapshot[applied={T|T < 7315832760802013184}]
I20260807 08:39:42.630225 11162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "54229f45723f46deadbd15943666dae5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } } }
I20260807 08:39:42.630355 11162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:42.631110 11137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "54229f45723f46deadbd15943666dae5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } } }
I20260807 08:39:42.631253 11137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:39:42.632766 11195 catalog_manager.cc:1347] Generated new cluster ID: 059f0375f4024a2fb61af9f116530acd
I20260807 08:39:42.632867 11195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:42.634965 11162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "54229f45723f46deadbd15943666dae5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } } }
I20260807 08:39:42.635136 11162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:42.636588 11217 mvcc.cc:205] Tried to move back new op lower bound from 7315832760802013184 to 7315832760765968384. Current Snapshot: MvccSnapshot[applied={T|T < 7315832760802013184}]
I20260807 08:39:42.636744 11192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "54229f45723f46deadbd15943666dae5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54229f45723f46deadbd15943666dae5" member_type: VOTER last_known_addr { host: "127.9.174.62" port: 36265 } } peers { permanent_uuid: "3b8749f0d6fb49afbdf3cfd3a7e8a7fb" member_type: VOTER last_known_addr { host: "127.9.174.61" port: 40989 } } peers { permanent_uuid: "7f1cfce791594fa786978723b085ec0c" member_type: VOTER last_known_addr { host: "127.9.174.60" port: 46185 } } }
I20260807 08:39:42.636838 11192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:39:42.646795 11129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.614274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60858 (local address 127.9.174.60:46185)
0807 08:39:42.614398 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.614404 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.635466 (+ 21062us) server_negotiation.cc:407] Connection header received
0807 08:39:42.635521 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.635524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.635583 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.635649 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.638422 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.639204 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.640298 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.640632 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.640699 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.642631 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:42.642635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:42.642638 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:42.642667 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:42.644664 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.645229 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.645233 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.645235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.645335 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.645681 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.645688 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.645690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.645973 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:42.646085 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:42.646470 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 08:39:42.646618 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 08:39:42.649278 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.609422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55002 (local address 127.9.174.61:40989)
0807 08:39:42.609580 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.609586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.620702 (+ 11116us) server_negotiation.cc:407] Connection header received
0807 08:39:42.627343 (+ 6641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.627349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.627433 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.627497 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.629389 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.630179 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.639574 (+ 9395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.639962 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.640047 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.645037 (+ 4990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:42.645043 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:42.645046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:42.645085 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:42.647076 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.647640 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.647645 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.647647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.647752 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.648109 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.648113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.648115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.648379 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:42.648536 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:42.648930 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 08:39:42.649101 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 08:39:42.653342 11195 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:42.654053 10920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:42.616273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38546 (local address 127.9.174.62:36265)
0807 08:39:42.616389 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.616394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.635686 (+ 19292us) server_negotiation.cc:407] Connection header received
0807 08:39:42.635798 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.635801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.635857 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.635945 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.637597 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.638388 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.642226 (+ 3838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:42.642762 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:42.642862 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:42.649372 (+ 6510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:42.649376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:42.649379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:42.649417 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:42.651496 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.652397 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.652401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.652404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.652512 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:42.652927 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:42.652931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:42.652933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:42.653205 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:42.653366 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:42.653726 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 08:39:42.653860 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":42}
I20260807 08:39:42.655182 11195 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:42.663509 11195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54229f45723f46deadbd15943666dae5: Generated new TSK 0
I20260807 08:39:42.666033 11195 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:39:42.708487 11031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:42.704961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55004 (local address 127.9.174.61:40989)
0807 08:39:42.705139 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.705145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.706868 (+ 1723us) server_negotiation.cc:407] Connection header received
0807 08:39:42.707304 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.707309 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.707385 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.707450 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.708291 (+ 841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55004: BlockingRecv error: recv got EOF from 127.0.0.1:55004 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 08:39:42.710397 11129 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:42.706034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60860 (local address 127.9.174.60:46185)
0807 08:39:42.706222 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.706230 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.709347 (+ 3117us) server_negotiation.cc:407] Connection header received
0807 08:39:42.709541 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.709545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.709608 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.709663 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.710230 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60860: BlockingRecv error: recv got EOF from 127.0.0.1:60860 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 08:39:42.713315 10920 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:42.711756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38562 (local address 127.9.174.62:36265)
0807 08:39:42.711910 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.711914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.712237 (+ 323us) server_negotiation.cc:407] Connection header received
0807 08:39:42.712416 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.712420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.712478 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.712532 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.713147 (+ 615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38562: BlockingRecv error: recv got EOF from 127.0.0.1:38562 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 08:39:42.718540 11031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:42.716873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55020 (local address 127.9.174.61:40989)
0807 08:39:42.717001 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.717006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.717442 (+ 436us) server_negotiation.cc:407] Connection header received
0807 08:39:42.717694 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.717698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.717799 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.717871 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.718416 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55020: BlockingRecv error: recv got EOF from 127.0.0.1:55020 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 08:39:42.724414 11031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:42.722661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.9.174.61:40989)
0807 08:39:42.722771 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.722775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.723451 (+ 676us) server_negotiation.cc:407] Connection header received
0807 08:39:42.723654 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.723658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.723715 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.723769 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.724295 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55034: BlockingRecv error: recv got EOF from 127.0.0.1:55034 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 08:39:42.730126 11031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:42.728578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55038 (local address 127.9.174.61:40989)
0807 08:39:42.728685 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.728689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.729205 (+ 516us) server_negotiation.cc:407] Connection header received
0807 08:39:42.729391 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.729394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.729447 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.729498 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.730005 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55038: BlockingRecv error: recv got EOF from 127.0.0.1:55038 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 08:39:42.737955 11031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:42.736326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55040 (local address 127.9.174.61:40989)
0807 08:39:42.736486 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.736490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.737028 (+ 538us) server_negotiation.cc:407] Connection header received
0807 08:39:42.737210 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.737214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.737271 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.737323 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.737832 (+ 509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55040: BlockingRecv error: recv got EOF from 127.0.0.1:55040 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260807 08:39:42.751124 11031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:42.748719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55046 (local address 127.9.174.61:40989)
0807 08:39:42.748869 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.748875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.749596 (+ 721us) server_negotiation.cc:407] Connection header received
0807 08:39:42.749842 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.749847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.749992 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.750055 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.750834 (+ 779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55046: BlockingRecv error: recv got EOF from 127.0.0.1:55046 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260807 08:39:42.773420 11031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:42.770942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55052 (local address 127.9.174.61:40989)
0807 08:39:42.771156 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.771162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.772066 (+ 904us) server_negotiation.cc:407] Connection header received
0807 08:39:42.772352 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.772357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.772442 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.772506 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.773238 (+ 732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55052: BlockingRecv error: recv got EOF from 127.0.0.1:55052 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
W20260807 08:39:42.811853 11031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:42.809170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55064 (local address 127.9.174.61:40989)
0807 08:39:42.809366 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.809386 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.810312 (+ 926us) server_negotiation.cc:407] Connection header received
0807 08:39:42.810634 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.810639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.810717 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.810787 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.811674 (+ 887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55064: BlockingRecv error: recv got EOF from 127.0.0.1:55064 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260807 08:39:42.880877 11031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:42.879163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55074 (local address 127.9.174.61:40989)
0807 08:39:42.879454 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:42.879462 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:42.879675 (+ 213us) server_negotiation.cc:407] Connection header received
0807 08:39:42.879920 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:42.879924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:42.879997 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:42.880109 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:42.880711 (+ 602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55074: BlockingRecv error: recv got EOF from 127.0.0.1:55074 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260807 08:39:43.013566 11031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:43.011937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55076 (local address 127.9.174.61:40989)
0807 08:39:43.012155 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:43.012161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:43.012354 (+ 193us) server_negotiation.cc:407] Connection header received
0807 08:39:43.012588 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:43.012592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:43.012659 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:43.012764 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:43.013400 (+ 636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55076: BlockingRecv error: recv got EOF from 127.0.0.1:55076 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 08:39:43.274533 11031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:39:43.273065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55084 (local address 127.9.174.61:40989)
0807 08:39:43.273271 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:43.273279 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:43.273378 (+ 99us) server_negotiation.cc:407] Connection header received
0807 08:39:43.273598 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:43.273613 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:43.273673 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:43.273778 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:43.274384 (+ 606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55084: BlockingRecv error: recv got EOF from 127.0.0.1:55084 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 08:39:43.597592 11158 catalog_manager.cc:1259] Loaded cluster ID: 059f0375f4024a2fb61af9f116530acd
I20260807 08:39:43.597725 11158 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb: loading cluster ID for follower catalog manager: success
I20260807 08:39:43.600005 11158 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb: acquiring CA information for follower catalog manager: success
I20260807 08:39:43.601543 11158 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3b8749f0d6fb49afbdf3cfd3a7e8a7fb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:39:43.623904 11209 catalog_manager.cc:1259] Loaded cluster ID: 059f0375f4024a2fb61af9f116530acd
I20260807 08:39:43.624010 11209 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c: loading cluster ID for follower catalog manager: success
I20260807 08:39:43.625926 11209 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c: acquiring CA information for follower catalog manager: success
I20260807 08:39:43.627341 11209 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7f1cfce791594fa786978723b085ec0c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:39:43.794742 11322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:43.789841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55088 (local address 127.9.174.61:40989)
0807 08:39:43.790287 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:43.790292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:43.790313 (+ 21us) server_negotiation.cc:407] Connection header received
0807 08:39:43.790488 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:43.790492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:43.790785 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:43.790932 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:39:43.791305 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:43.791972 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:43.792897 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:43.793173 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:43.793261 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:43.793630 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:39:43.794244 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:39:43.794348 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:43.794416 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 08:39:43.794483 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":180,"threads_started":1}
I20260807 08:39:43.803243 11331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:43.798767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60866 (local address 127.9.174.60:46185)
0807 08:39:43.799099 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:43.799105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:43.799124 (+ 19us) server_negotiation.cc:407] Connection header received
0807 08:39:43.799265 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:43.799269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:43.799436 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:43.799606 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:39:43.799859 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:43.800448 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:43.801376 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:43.801641 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:43.801764 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:43.802144 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:39:43.802745 (+ 601us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:39:43.802903 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:43.802939 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 08:39:43.803012 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":78,"threads_started":1}
I20260807 08:39:43.811898 11322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:43.807200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55094 (local address 127.9.174.61:40989)
0807 08:39:43.807386 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:43.807391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:43.807408 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:39:43.807625 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:43.807628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:43.807715 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:43.807765 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:39:43.808231 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:43.808908 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:43.810489 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:43.810831 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:43.810940 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:43.811276 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:39:43.811376 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:39:43.811464 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:43.811619 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 08:39:43.811691 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 08:39:43.812443 11331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:43.806978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60876 (local address 127.9.174.60:46185)
0807 08:39:43.807153 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:43.807158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:43.807416 (+ 258us) server_negotiation.cc:407] Connection header received
0807 08:39:43.807598 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:43.807602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:43.807659 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:43.807709 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:39:43.808090 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:43.808925 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:43.811032 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:43.811335 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:43.811450 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:43.811743 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:39:43.811848 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:39:43.811895 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:43.812203 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 08:39:43.812266 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 08:39:43.812520 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:43.806888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38576 (local address 127.9.174.62:36265)
0807 08:39:43.807730 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:43.807742 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:43.807756 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:39:43.807862 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:43.807865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:43.808051 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:43.808195 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:39:43.808461 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:43.809209 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:43.810237 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:43.810579 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:43.810684 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:43.811210 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:39:43.811691 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:39:43.811783 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:43.811933 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 08:39:43.812013 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":579,"threads_started":1}
I20260807 08:39:43.816612 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 10819
I20260807 08:39:43.833611 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 10924
I20260807 08:39:43.849726 9912 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1t_l5C/build/debug/bin/kudu with pid 11032
2026-08-07T08:39:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3565 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (7207 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (328103 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST